Laboratory Statistics, Method Development, and Quality Control PDF

Laboratory Statistics, Method Development, and Quality Control CHAPTER 3 Preamble PowerPoints are a general overview and are provided to help students take notes over the video lecture ONLY. PowerPoints DO NOT cover the details needed for the Unit exam Each student is responsible for RE...

Laboratory Statistics, Method Development, and Quality Control CHAPTER 3 Preamble PowerPoints are a general overview and are provided to help students take notes over the video lecture ONLY. PowerPoints DO NOT cover the details needed for the Unit exam Each student is responsible for READING the TEXTBOOK for details to answer the UNIT OBJECTIVES Unit Objectives are your study guide (not this PowerPoint) Test questions cover the details of UNIT OBJECTIVES found only in your Textbook! Statistics Statistics Mathematical process of dealing with the collection, analysis, interpretation, and presentation of masses of laboratory numerical data Descriptive Statistics Includes the mean, range, variability and distribution of a data set. Commonly used in the laboratory. Inferential Statistics Concerned with the relationship among different sets or samples of data. Example: Comparing the mean of one set of data to another set of data Laboratory Statistics Statistics should be viewed as tools that are available for the laboratory staff to use. Knowing which tool or statistic to use is important. Computation of most statistics is performed using computer software packages or calculators. Laboratory Statistics Measures of Central Tendency Mean: average Median: middle point Mode: most frequently occurring value Laboratory Statistics Measures of Dispersion Range (r) Measure of spread or variation in a set of data The difference between the largest and the smallest numbers of the data set Standard Deviation (s) Measures dispersion around the mean Shape of distribution on a Gaussian curve (bell curve) Coefficient of Variation (CV) Measures relative standard deviation divided by the mean Defined as 100 times the standard deviation s CV = ´ 100 x Laboratory Statistics Standard Deviation Is the square root of the variance of the values in any one observation or in a series of results Standard Deviation or SD = the distribution around the average for a particular group 1 SD says 68% of all values in a measured group will fall into this area 2SD says 95% of all values in a measured group will fall into this area 3SD says 99.7% of all values in a measured group will fall into this area Laboratory Statistics Standard Deviation At least 30 patient’s specimens are selected and test is run These values are averaged Standard deviations are figured by the following formula: S= ∑ (x1 – x)2 N–1 X = each individual value X = mean S = Standard Deviation ∑ = Sum of N = number of tests performed = square root Laboratory Statistics Coefficient of Variation (CV’s) CV’s are used for comparison’s To compare one method to another method To compare one lab to another lab Formula: CV% = 1 SD X 100 Mean CV for 1st method CV for 2nd method CV = 4 X 100 CV = 2 X 100 87 87 CV = 0.045 X 100 CV = 0.023 X 100 CV = 4.5% CV = 2.2% The closer the CV value is to “1” the better the precision. Laboratory Statistics Measures of Dispersion Variance (sd2) Calculated by squaring the standard deviation Can also be derived by subtracting the mean from each of the values, squaring the resulting differences, and then adding up the squared differences. Outliers Measurement that belongs to a population other than the one to which most of the measurements belong. Can distort the computed values of statistics Can cause incorrect inferences to be made about the population parameters of interest Laboratory Statistics Population distribution Normal distribution of analytes in a selected population is a: Continuous distribution Symmetric around the mean (σ 2 ). Predictably related to the standard deviation (sd), or sigma (σ), and variance Laboratory Statistics Confidence Intervals Keep in mind that we expect 1 out of 20 to be out of the 2 SD on either side of the Gaussian curve This means that we expect 2.5 out of 100 to be above the 2 SD range and 2.5 out of 100 to be below the 2 SD range 2.5 out of 100 is the same as 1out of 40 to be high and 1 out of 40 to be low Therefore, 95% of normal patient’s values will fall in the 2 SD Range 95% confidence interval is also used to explain the day-to-day shifts in values for a particular analytic procedure If we expect 95% to be within “normal” limits that means we expect 5% to be normal and outside the 2 SD range 5 out of 100 is the same as 1 out of 20 Laboratory Statistics Regression Regression analysis is useful in assessing specific aspects of the relationship between variables Ultimate objective is to predict or estimate the value of one variable based on a given value of the second variable. Regression analysis is commonly used in the comparison of two methods or two instruments and to evaluate the linearity of an instrument or method. Correlation Correlation statistics measure the strength of the relationship between variables. Analytical Performance Parameters Accuracy Limit of Detection Precision Interference Types of Errors Stability Linearity Ruggedness Analytical Range Analytical Sensitivity Robustness Analytical Specificity Performance standards Analytical Performance Parameters Accuracy The closeness of the agreement between the measured value of an analyte to its “true” value. Can be evaluated using proficiency testing samples. Recovery experiments using the method of addition are also used to determine the accuracy of a method. Precision Ability to produce the same value for replicate measurements of the same sample Also described as the random variation in a population of data Analytical Performance Parameters Types of Errors Random Occurs without prediction or regularity Systematic Error that is consistently low or high Constant Error is consistently low or high by the same amount over the entire concentration range. Proportional Error is consistently low or high by an amount propor- tional to the concentration of the analyte. Discordant results Commonly used to describe laboratory results that do not agree false negative, false positive, errors, and inaccurate results Provide C L Ss with one of their greatest challenges because such results often occur sporadically and may occur with some specimens but not others Analytical Performance Parameters Linearity Quality or state of being linear. If the plot is linear, the range tested is termed the linear range of the method. Analytical Range Range of concentration in the sample over which the method is applicable without “modification” Should be wide enough to include 95 to 99% of the expected samples without predilution. Analytical Sensitivity The slope of the calibration curve and the ability of an analytical procedure to produce a change in the signal for a defined change of the analyte quantity Ideally a method should have a high level of ana- lytical sensitivity and a low detection limit. Analytical Specificity Ability of a method to measure only the analyte it claims to measure without reacting with other related substances Analytical Performance Parameters Limit of Detection Represents the lowest concentration or quantity of an analyte that significantly exceeds the measurement of a blank sample Depends on the amplitude of the blank value and must be precise at that level. Interference Effects of a compound(s) other than the analyte being measured Stability Stability of reagents, calibrators, and controls must be investigated. Impacts the laboratory in both cost and efficiency Ruggedness Ability of the assay to perform in a consistent, reliable fashion when used by different operators and with different batches of reagents over an extended period of time. Analytical Performance Parameters Robustness Capacity of a method to remain unaffected by small, deliberate variation in method parameters Performance Standards Need to be established before any analytical experiments are begun Focus is error. Reference Interval CLSI has published a document that provides guidelines on how to define and determine reference intervals in the clinical laboratory. Experimental Phase of Method Evaluation Tasks performed and the order in which they are performed vary considerably from laboratory to laboratory. Comparison of methods experiment involves measuring patient specimens by both existing (reference) and new (test) methods Clinical Decision Limits Diagnostic Tests Screening Application of a test to individuals who have not yet exhibited any clinical symptoms in order to classify them with respect to their probability of having a particular disease Screening tests are not always infallible. May yield a false positive or a false negative result Following issues of probability should be considered in evaluating the usefulness of test results: 1. Given that a subject has the disease, what is the probability of a positive test result (or the presence of symptoms)? 2. Given that a subject does not have the disease, what is the probability of a negative test result (or the absence of symptoms)? 3. Given a positive screening test result (or the presence of a symptom), what is the probability that the subject has the disease? 4. Given a negative screening test result (or the absence of a symptom), what is the probability that the subject does not have the disease? Clinical Decision Limits Diagnostic Tests Sensitivity Specificity Predictive value of a positive and negative test Diagnostic Parameters Associated With Predictive Value Theory Diagnostic sensitivity = TP/ TP + FN Diagnostic specificity = TN/ FP + TN blank Number of Number of Total Subjects with Subjects with Positive Test Negative Test Results Results Number of Subjects TP FN TP + FN with Disease Number of Subjects FP TN FP + TN without Disease Totals TP + FP FN + TN TP + FP + TN + FN TP, True positives (number of diseased patients correctly classified by the test) FP, False positives (number of patients without the disease misclassified by the test) FN, False negative (number of diseased patients misclassified by the test) TN, True negative (number of patients without the disease correctly classified by the test) Reference Range (Normal Range) Interval between and including two reference limits Reference limit is a numerical value(s) derived from the reference distribution. Reference Denotes a well-defined selection of subjects used to mathematically determine the numerical values equivalent to reference limits and thus reference interval Reference intervals are derived from a reference individual. Several factors must be addressed before determining a reference interval: Selection of reference individuals Preanalytical variables Analytical methods Statistical applications Selection of reference individuals must be determined before any data are collected. Data should be collected from “normal,” healthy individuals. List of selection criteria must be established. Another set of criteria that require attention is referred to as partitioning criteria. Reference Range (Normal Range) Preanalytical variables Include factors that affect the reference individual, specimen collection, and specimen handling Diet Fasting versus nonfasting Drug therapies Physical activity Stress Time Body posture Site preparation Equipment Technique Transport Storage Clotting Separation of serum or plasma Preparation for analysis Reference Range (Normal Range) Analytical methods Factors that affect analytical performance, including equipment, reagents, calibrators, and calculations, require control and documentation. Quality-control materials should be assayed throughout the reference interval study to monitor the analytical procedure. Statistical applications The reference interval may be described as the interval between the 2.5th (lower reference limit) and 97.5th (upper reference limit) percentiles of a group of data obtained from a reference population. CLSI guidelines recommend: A minimum of 120 reference values be used Reference interval be determined by the nonparametric method Quality Assurance and Quality Control Quality Assurance Total quality management (TQM) CLSs have incorporated many of the aspects of TQM. Quality assurance (QA) in a health-care facility: Represents global issues Responsibility of everyone involved in the care of patients Many essentials including: Commitment Facilities Resources Competent staff Reliable procedures, methods, and instrumentation Quality Control Refers to procedures for monitoring and evaluating the quality of the analytical testing process of each method to ensure the accuracy and reliability of patient test results and reports. Purpose is to verify the stability and accuracy of calibration and testing systems. Quality Control Quality-Control Materials Best practice when selecting appropriate material for Q C is to use a matrix that is similar to the test specimens. Several factors to consider: The materials must be stable. The materials must be available in aliquots or vials. The materials can be analyzed periodically over a long span of time. There is little vial-to-vial variation. The concentration of analyte should be in the normal and abnormal ranges. Target Values Assayed Q C material has target values are determined by the manufacturer. Unassayed Q C material has no predetermined target values. Ranges must be determined by the laboratory. Target values include the mean and standard deviation of the analyte for the particular control material. Quality Control Quality-Control Limits Establishment is not an easy task. Control limits Must fall within the total allowable error of the method Must allow for successful completion of proficiency testing required by state and/or federal regulatory agencies Levey-Jennings Control Charts Show the difference between the observed values and the expected mean Created by calculating the mean concentration and up to ±3s for a pool of QC material A trend is a pattern of data in which all of the QC values continue to increase or decrease over a period of time. A shift is consecutive data that remain on one side of the mean for a period of time. Quality Control - Shifts and Trends Shift - Four consecutive control values above or below the mean. +2SD +1SD Mean -1SD -2SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Trend - Four consecutive control values moving in one direction Quality Control Levey-Jennings Chart  Used to monitor controls for determining if patient results can be REPORTED or not  Each time the controls are run the value is placed on the Levy-Jennings Chart ▪ The values are expected to fall up and down around the mean ▪ A value may fall outside the range 5% of the time or 1 out of 20 runs ▪ A value may fall above the mean 2.5% of the time or 1 out of 40 runs ▪ A value may fall below the mean 2.5% of the time or 1 out of 40 runs ▪ Evaluation of QC data is critical to accurate test results ▪ Problems may arise to indicate corrective action needs to be taken ▪ Shifts, Trends, values outside of acceptable QC limits – Westgard Rules Quality Control Power Functions Also called power curves Useful for: 1. Evaluating the performance capabilities of individual control procedures 2. Comparing the performance of different control procedures 3. Designing a new procedure with improved performance characteristics Westgard Multirule Procedures Requires a chart in which lines for control limits are drawn at the mean This chart can be adapted to existing L-J charts by adding one or two sets of control limits. Table 3-10 Westgard Quality-Control Rules and Interpretation (1 of 2) Control Rule Interpretation Designations 2 sub 2 s 12s One control observation exceeding the mean± 2s -this rule may be Plus or minus 2 s used as a warning rule. 1 sub 3 s 13s One control observation exceeding the mean± 3s -recommend Plus or minus 3 s rejecting patient results; this rule is sensitive to random error. 2 sub 2 s Two consecutive control observations exceeding the same mean plus 22s 2s or mean minus 2s limit-recommend rejecting patient results; this rule is sensitive to systematic error. R sub 4 s One observation exceeding the mean plus 2s and another exceeding R 4s the mean minus 2s within a run not between runs-recommend rejecting patient results; this rule is sensitive to random error. Note: standard deviation (s). Table 3-10 Westgard Quality-Control Rules and Interpretation (2 of 2) Control Rule Interpretation Designations 4 sub 1 s Four consecutive observations exceeding the mean 41s plus 1s or the mean minus 1s-recommend rejecting patient results; this rule is sensitive to systematic error. 10 to the power of x-bar Ten consecutive control observations falling on one side of the mean (above or below, with no other 10 x requirement on size of the deviations)-recommend rejecting patient results; this rule is sensitive to systematic error. 7 sub T 7T Reject run whenever control measurements trend in the same direction (i.e., get progressively higher or lower) Note: standard deviation (s). Figure 3-11 Quality-control Rule Violations (1 of 6) Figure 3-11 Quality-control Rule Violations (3 of 6) Figure 3-11 Quality-control Rule Violations (5 of 6) Quality Control Sources of Random and Systematic Errors Sources of error vary depending on whether they are random or systematic. Random Error Systematic Error Operator technique Improper alignment of sample or reagent pipettes Use of non-reagent-grade water Unstable incubator chambers Incorrect reconstitution of control material Change of reagent lot Power supply Change in calibrator lot Pipetting mistakes Deterioration of reagents in use Automated pipette problems Deterioration of control material while in use Air bubbles in tubing Evaporation of sample during analysis Dirty filter or gradual delaminating of monochromatic filter Change in test operator Recent calibration Deteriorating light source Incorrect handling of control material Quality Control Detecting Quality-Control Problems Computers Advantages 1. Real-time review 2. Early detection of QC problems 3. Documentation of the QC process Patient Results Monitoring patient results, especially serial results on a single patient, can alert the CLS to quality issues that require attention. Steps to Remedy Westgard Rule Violations If QC results violate any Westgard rules, the laboratory must have a procedure that will resolve these violations. Lesser-Used Quality-Control Applications Several QC applications available for use by the laboratory staff, and these can be reviewed in the references cited. Quality Control Evaluating Patient Results Whenever laboratory results are generated on a patient specimen, the C L S is faced with at least three options: 1. Accept the result 2. Reject the result 3. Modify the result Delta checks Procedure in which the C L S compares two consecutive laboratory results on a patient Limit checks Represent laboratory results that may represent serious conditions relative to the patient Quality Control Evaluating Patient Results Autoverification Verification and release of patient results using software-based algorithms with decision-making logic via the L I S Proficiency testing Also referred to as surveys An example of external quality control wherein an agency or organization provides biological samples whose concentrations are unknown to the testing clinical laboratory. Calibration Purpose is to “substantiate the continued accuracy of the test system throughout the laboratory reportable range of the test results for the test system.” Quality Control Materials To ensure clinically correct results, laboratories use the following: Standards - Highly purified substance of a known composition Control samples - Specimen that is similar to the patient's blood with a known concentration of constituent. Proficiency testing programs This is a program where samples are sent to a group of laboratories for analysis. Results are then compared with the other labs. This testing process is included as a component of the quality assurance program. Quality Control Material Control specimen Similar in composition to the unknown specimen Is included in every batch or run It must be carried through the entire test procedure It must be treated in the exact same way as the patient specimen It is affected by any or all variables that affect the unknown (patient) specimen In controlling reliability of the laboratory results, the objective is to: Reject results when there is evidence that more than the permitted amount of error has occurred. Several ways of doing this: Run a control with every batch ( a collection of any number of specimens to be analyzed) or run Standard solutions Control specimens Duplicates Quality Control Material Chemistry controls are purchased, and the same lot number is used for about a year. When the lot number is scheduled to expire New control data must be collected for about a month prior to expiration date A new mean and 2 SD range must be established for the new controls Controls are reconstituted with water, mixed, and allowed to come completely into solution Date of reconstitution, date of expiration, and initials of the tech who reconstituted must be on the control bottle. Quality Control Material Controls must be run at room temperature Controls must be kept closed and in the refrigerator between runs Collection of data Must mimic all conditions found when actually running the controls during the year When controls are new and old When instrument is clean and dirty By each and every tech who runs the instrument Must be run over a period of time Must use at least 30 data points but the larger the group of data the better the mean and range will be Quality Control Implementation of new QC is a process of running the “old” lot number of control to determine if patient’s results can be reported while collecting data for “new” lot number. There are three control values for each test performed in the lab to make sure all possible areas of patient’s results are covered Abnormal High Normal Abnormal Low Automated Instrumentation today collects the data and calculates the mean and 2SD range Let’s Practice Together  QC Values and Calculation of Standard Deviation Calculate the SD and create a levy-Jennings chart Test Value (mg/dL) Minus the Mean Deviation from Mean Deviation squared (X- X X (X – X) X)2 82 87 -5 25 85 87 -2 4 90 87 2 4 86 87 -1 1 91 87 3 9 90 87 2 4 81 87 -6 36 86 87 -1 1 94 87 6 36 89 87 1 1 Sum 874 Sum 121 Calculations Mean X = sum of X SD = 121 9 n X = 874 = 87.4 10 13.4 SD = X = 87 (round off) Standard Deviation SD = 3.7 or SD = SD = 4 (round off) ∑ (X-X)2 SD = 4 N-1 2 SD = 8 Postamble READ the TEXTBOOK for the details to answer the UNIT OBJECTIVES. USE THE UNIT OBJECTIVES AS A STUDY GUIDE All test questions come from detailed material found in the TEXTBOOK (Not this PowerPoint) and relate back to the Unit Objectives