Diagnostic Accuracy & Reference Values (Clinical Chemistry) PDF

PH 166: CLINICAL CHEMISTRY 1 Diagnostic Accuracy and Concept of Reference Values Asst. Prof. Raycha Lei Concess M. Rama-Sabandal | January 30, 2025 OUTLINE A.​ Diagnostic Accuracy a.​ Accuracy vs. Precision b.​ Concepts in Di...

PH 166: CLINICAL CHEMISTRY 1 Diagnostic Accuracy and Concept of Reference Values Asst. Prof. Raycha Lei Concess M. Rama-Sabandal | January 30, 2025 OUTLINE A.​ Diagnostic Accuracy a.​ Accuracy vs. Precision b.​ Concepts in Diagnostic Accuracy i.​ Gold Standard ii.​ Four Groups in the 2x2 Truth Table iii.​ Overall Accuracy c.​ Sensitivity and Specificity Figure 2. Distribution of Test Results with Perfect Diagnostic Accuracy. d.​ Predictive Values B.​ Concept of Reference Values ​ In reality, an overlap between populations at both low a.​ Background and high cut-offs exists b.​ Choosing a Reference Interval i.​ Ways to Choose a Reference Interval ○​ Arbitrary cut-off points is the reason for the repetition c.​ Influencing Factors of tests to ensure the accuracy of results (BSPH 2025) d.​ Establishing Reference Interval ○​ It is recommended for physicians to not base on the e.​ Limitations laboratory results, but look at the whole picture C.​ Review Questions ​ Even if you have an idea, do not conclude D.​ References E.​ Appendices SUMMARY OF EQUATIONS/INFO (if applicable) OVERALL ACCURACY (𝑇𝑁 + 𝑇𝑃) 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (𝑇𝑁 + 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁) SENSITIVITY Figure 3. Realistic Distribution of Test Results. 𝑇𝑃 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑇𝑃 + 𝐹𝑁 ​ The test’s diagnostic accuracy is determined by comparing its ability to discern true disease from SPECIFICITY non-disease as determined by a diagnostic gold standard 𝑇𝑁 (i.e. truth) 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 = 𝑇𝑁 + 𝐹𝑃 ​ Patients can be classified into four groups in a 2x2 table or “Truth Table” based on the results from the test and the POSITIVE PREDICTIVE VALUE (PPV) gold standard 𝑇𝑃 𝑃𝑃𝑉 = 𝑇𝑃 + 𝐹𝑃 ○​ X-axis: Gold standard ​ Diseased or Non-diseased NEGATIVE PREDICTIVE VALUE (NPV) ​ Confirmatory Test 𝑇𝑁 ○​ Y-axis: Test performance 𝑁𝑃𝑉 = 𝑇𝑁 + 𝐹𝑁 ​ Test positive or Test negative ​ Novel Test (Screening) DIAGNOSTIC ACCURACY ACCURACY VS. PRECISION ​ Accuracy ○​ Closeness of a measurement to its true value ​ Precision ○​ Closeness of agreement among a set of results ​ It is ideal to achieve high accuracy and precision for every test Figure 4. 2x2 Truth Table Format. Gold Standard ​ An experimental model that has been thoroughly tested Figure 1. Differences between Accuracy and Precision. and has a reputation in the field as a reliable method ​ Suggest that a test provides authoritative, and CONCEPTS IN DIAGNOSTIC ACCURACY presumably indisputable evidence that a condition does ​ A test with perfect diagnostic accuracy could determine or does not exist the presence or absence of disease with certainty ​ Have increasingly been referred to as reference ○​ The established cutoff point would perfectly separate standards diseased from non-diseased populations in an ideal ○​ Thus removing what seemed to be unreserved world endorsement Four Groups in the 2x2 Truth Table ​ True Results #MagkabigkisBenteSais Group 3 | 1 of 6  ○​ Non-overlapping areas of the two population SAMPLE PROBLEM 1 distributions In this evaluation study of a hypothetical cardiac marker, 200 ○​ True-Positives (TP) patients with acute myocardial infarction (AMI) and 200 ​ Those correctly classified as abnormal healthy subjects are recruited for a study designed to mimic a ​ With the disease prevalence of 50%. The assay is performed and is compared ○​ True-Negatives (TN) with a “gold standard” test for AMI, and the 2x2 truth table ​ Those correctly classified as normal shown here is generated. Compute for the measures of ​ Without the disease diagnostic accuracy. ​ False Results ○​ Overlapping areas of two populations, since a test cannot completely discriminate all abnormal patients Result Disease No Disease Total from normal ones Positive 196 20 216 ○​ False-Negatives (FN) ​ Those incorrectly classified as normal Negative 4 180 184 ○​ False-Positives (FP) ​ Those incorrectly classified as abnormal Total 200 200 400 ​ True positives can be classified as false negatives, while true negatives can be classified as false positives Solution: 𝑇𝑃 196 1.​ Sensitivity % = = 196 + 4 × 100 = 98% Overall Accuracy 𝑇𝑃 + 𝐹𝑁 𝑇𝑁 180 2.​ Specificity% = 𝑇𝑁 + 𝐹𝑃 = 180 + 20 × 100 = 90% ​ The overall accuracy of a test can then be defined as the 𝑇𝑃 196 proportion of true classifications out of all 3.​ PPV % = 𝑇𝑃 + 𝐹𝑃 = 196 + 20 × 100 = 91% classifications 4.​ 𝑇𝑁 180 NPV % = 𝑇𝑁 + 𝐹𝑁 = 180 + 4 × 100 = 98% Formula 1. Overall Accuracy 5.​ (𝑇𝑁 + 𝑇𝑃) (180 + 196) Accuracy = (𝑇𝑁 + 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁) = (180 + 196 + 20 + 4) = 94% (𝑇𝑁 + 𝑇𝑃) 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (𝑇𝑁 + 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁) SAMPLE PROBLEM 2 SENSITIVITY AND SPECIFICITY A screening blood test for Hepatitis B virus was compared ​ Sensitivity against the detection of HPV D and E (gold standard) in the ○​ Proportion of people with inherent disease who test liver. Complete the lacking information in the truth table and positive (true-positive) compute for the measure of diagnostic accuracy. ○​ Sensitive test - detects all or almost all diseased individuals; also used to screen for more contagious or infectious diseases, or where there is benefit in Result Disease No Disease Total having early detection and treatment Positive 120 8 (B) Formula 2. Sensitivity 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑇𝑃 Negative 15 80 95 𝑇𝑃 + 𝐹𝑁 Total 135 (A) (C) ​ Specificity ○​ Proportion of people without the disease who test Solution: negative (true-negative) 1.​ A = 𝐹𝑃 + 𝑇𝑁 = 80 + 8 = 88 ○​ Specific test - provides negative results for all or 2.​ B = 𝑇𝑃 + 𝐹𝑃 = 120 + 8 = 128 almost all disease-free individuals; used when 3.​ C = 128 + 95 = 223 subdued procedures are expensive or high-risk 𝑇𝑃 120 4.​ Sensitivity % = 𝑇𝑃 + 𝐹𝑁 = 120+15 × 100 = 89% Formula 3. Specificity 𝑇𝑁 80 𝑇𝑁 5.​ Specificity% = 𝑇𝑁 + 𝐹𝑃 = 80 + 8 × 100 = 91% 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 = 𝑇𝑁 + 𝐹𝑃 𝑇𝑃 120 6.​ PPV % = 𝑇𝑃 + 𝐹𝑃 = 120 + 8 × 100 = 94% 𝑇𝑁 80 7.​ NPV % = 𝑇𝑁 + 𝐹𝑁 = 80 + 15 × 100 = 84% PREDICTIVE VALUES (𝑇𝑁 + 𝑇𝑃) (120 + 80) ​ Positive Predictive Value (PPV) 8.​ Accuracy = (𝑇𝑁 + 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁) = (80 + 120 + 8 + 15) = 90% ○​ Probability of patients with true positive results (have the disease/condition of interest) to test positive SAMPLE PROBLEM 3 Formula 4. Positive Predictive Value (PPV) A certain test for analyte X has a reference value set at ≤6.50 𝑇𝑃 𝑃𝑃𝑉 = 𝑇𝑃 + 𝐹𝑃 mg/dL. A researcher decided to adjust the significant figure of the result from 2 decimal places to 1 (from 6.50 to 6.5). What false results are prone to increase and which between ​ Negative Predictive Value (NPV) sensitivity and specificity will it affect? ○​ Probability of patients with true negative results (no disease) to test negative Solution: Formula 5. Negative Predictive Value (NPV) ​ ↑ False Negative, ↓ Sensitivity 𝑇𝑁 𝑁𝑃𝑉 = 𝑇𝑁 + 𝐹𝑁 ​ Example: (A) 6.50 cut-off; (B) 6.5 cut-off ○​ 6.49 will be rounded off to 6.5 but is equal to the cut-off so it is still negative ○​ 6.51 in A is already positive but will be negative to B when it could be rounded off, thus having high false negative results PH 166 | Trans Title 2 of 6  professionals, the more commonly known term is ​ Diagnostic accuracy of a test tends to diminish as the test “reference range” becomes more widely used in population ​ Initial validation studies are often performed on a small ​ Normal or Reference? Value, Values, Limit, Interval, or group of individuals in whom disease is clearly absent or Range? It seems all terms are accepted and used. present ​ However, in practice, patients exhibit a spectrum of illness, Ways to Choose a Reference Interval including early or mild disease, which tends to overlap with 1.​ Derive reference ranges from existing patient data non-diseased individuals, including those with other ​ Retrospective diseases, some case of which might cause abnormal test 2.​ Cite reference ranges from test manufacturers or other results laboratories ​ Thus, in clinical practice, the proportion of false test results ​ Cost-effective is higher than is claimed by the manufacturer (from more 3.​ Test a pool of perceived normal and healthy individuals limited studies in healthy individuals) ​ Ideally what we want to use; prospective ​ ↑ Accuracy (i.e., sensitivity and specificity), ↑ Predictive Value of a Test INFLUENCING FACTORS CONCEPT OF REFERENCE VALUES Table 1. Influencing Factors. INFLUENCING FACTORS (BSPH 2025) BACKGROUND Factor Example ​ Introduced in 1969 by R. Grasbeck and N.E. Saris Values can differ between a Age ○​ To describe fluctuations of blood analyte child and an elderly concentrations in well-characterized groups of International ranges may not individuals Genetic Background always apply due to differences in sex and race ○​ To replace the more ambiguous concept of High altitude = higher levels “normal values” which was usually incorrect or of hemoglobin inadequate Exposure to environmental ○​ To “establish a well-defined nomenclature and factors Exposure to sunlight = higher recommended procedures in the field” levels of Vitamin D ○​ The term “reference values” was used from 1987 to Different populations may 1991 need different reference Reference ranges differ from ​ It has since gained universal acceptance as one of the ranges for certain Caucasians and Asians most powerful tools in laboratory medicine to aid in the laboratory analytes clinical decision-making process Some samples may need a ○​ Has the power to direct individuals when to isolate, container with an inert material (i.e. glass) change diet, include maintenance, etc., thus, it is Sample collection container important to have well-defined references Materials such as plastic may ​ After which, there were guidelines (although minimal) made react with the analyte available on how to conduct studies establishing reference Some samples need Sample transport values to be iced ​ In 1993, the Clinical Laboratory Standard Institute Time interval specimen Examples are drug and (CLSI) and the International Federation of Clinical collection and analysis arterial blood gas test Chemistry and Laboratory Medicine (IFCC) published Could be refrigerated or Sample storage before the guideline, “Defining, Establishing, and Verifying analyzed immediately after analysis Reference Intervals in the Clinical Laboratory” collection ​ In this first publication, there was a clear distinction between healthy reference values measured in: ESTABLISHING REFERENCE INTERVAL ○​ Healthy populations or individuals (healthy) and; ​ From Jones and Barker (2008) ○​ Patient reference values measured in patients having various diseases (diseased) 1. Define the analyte for which the reference interval is ​ It is now commonly accepted that reference values being established, the clinical utility, biological variation describe fluctuations observed in healthy populations or and major variations in form individuals ​ Within- and between-person biological variability data ○​ Makes the definition of health or characterization of is important health status a critical step ​ Are there other biological forms or derivatives of the analyte? CHOOSING A REFERENCE INTERVAL ○​ Is there a difference between males and females? ○​ When sitting or lying down ​ Reference Interval ○​ At night or morning ○​ Set of values that includes upper and lower limits of a ○​ Example: Prolactin has varying forms (serum prolactin laboratory test based on a group of otherwise healthy or macroprolactin) people ​ Is it desirable or possible to exclude samples containing ○​ Values in between those limits may depend on such this derivative form from the interval setting process? factors as age, sex, and specimen type (blood, urine, ○​ Preparatory step spinal fluid, etc.) and can also be influenced by ​ Include the units to be used and corresponding conversion circumstantial situations such as fasting and exercise factors ○​ Though the term "reference interval" is usually the ○​ Conventional & SI units term preferred by laboratory and other health PH 166 | Trans Title 3 of 6 2. Define the method used, the accuracy base, and ○​ Example: Prostate-specific antigen (PSA) level of 4 analytical specificity ng/mL is often used to distinguish patients who ​ There should be a description of the method used in the require no further follow-up (“normal”) from those who laboratory, the accuracy base used for that assay, and require a prostate biopsy (“abnormal”) relevant issues regarding analytical specificity and, if ○​ If PSA = 3.5 ng/mL → low → no risk for prostate possible, evidence that the assay is working as specified cancer by the manufacturer and remains stable over time C.​ Upper Reference Limit ​ Accuracy base may be described as traceability to a ○​ Example: An increased cardiac troponin concentration method or reference material and ideally this would be is defined as a value exceeding the 99th percentile of expressed in a quantitative manner a normal reference population (American and ​ Standard Reference Materials European Cardiology Associations) ○​ Used in the clinical laboratory setting since biologic D.​ Therapeutic Target Ranges constituents are not available in very high purities ○​ For other analytes (e.g., cholesterol/lipids), laboratories ○​ Developed by National Institute of Standards and frequently provide therapeutic target ranges that Technology (NIST) in the U.S represent recommendations based on clinical trials and/or epidemiologic studies (Grundy et al, 2004) ○​ This is what we want (e.g. cholesterol, lipids) 5. Describe the data source(s), including: number of subjects, nature of subjects, exclusions, pre-analytical factors, statistical measures, outliers excluded and analytical method ​ The subjects being tested (the reference population) Figure 5. Example of Standard Reference Materials. should be as similar as possible to that for which the test will be applied, with the exception of the presence of 3. Define important pre-analytical considerations together disease with any actions in response to the interference ​ Some tests may be significantly different due to racial or ​ Pre-analytical factors/variables are factors that might environmental factors, so the main effects to consider influence laboratory results are age, sex and common factors such as obesity or ○​ Example: You want to analyze an analyte in blood but diabetes it also interacts with other factors–these factors should ​ It may be appropriate to partition (or to group) for these also be considered factors but consideration should be given to the population ○​ Includes patient preparation, coagulant, glassware where the test is likely to be used ​ Extensive literature review should be done ○​ Example: Iron content in blood differs by gender ○​ Done in order to learn about the factors (e.g. what will ​ All factors should be similar except the analyte happen if the patient exercises prior to collection) ​ A sample questionnaire for recruiting reference population ​ Examples: can be found on Appendix A ○​ Type of specimen like serum versus heparin plasma for measuring potassium 6. Define considerations of partitioning based on age, sex, ○​ Time of day for collection for serum cortisol or etc testosterone ​ Partitioning of a reference interval is the use of separate ​ Cortisol is high after waking up & after 30 minutes intervals for different sub-populations it declines → recommend to be an in patient ​ Necessary to take into account sex, age (pediatric and ​ Testosterone is higher in the morning than in the geriatric populations), reproductive status (puberty, afternoon → advise patient to come in early for menstrual cycle, stage of pregnancy, menopause) and race sample collection ​ The type of sample (e.g. plasma vs. whole blood, or ○​ Sample handling, such as time until centrifugation for random vs. first-morning spot urine sample) may also potassium measurement require separate reference intervals ​ High RPM or prolonged centrifuging can cause hemolysis of blood leaking potassium → 7. Define the number of significant figures, i.e. the degree increases levels reported of rounding ○​ Common interferences, such as hemolysis for ​ The number of significant figures of the results can potassium, CK, AST and LD markedly influence the impact of a reference interval or decision point 4.​ Define the principle behind the reference interval ​ For example, reporting HDL cholesterol to the nearest 0.1 A.​ Range of values into which 95% of non-diseased mmol/L with a decision point of