Quality Assessment and Quality Control in Clinical Labs PDF

6/26/2024 Chapter 3 Quality Assessment and Quality Control in the Clinical Laboratory Preamble PowerPoints are a general overview and are provided to help students take notes over the video lecture ONLY. PowerPoints DO NOT cover the details needed for th...

6/26/2024 Chapter 3 Quality Assessment and Quality Control in the Clinical Laboratory Preamble PowerPoints are a general overview and are provided to help students take notes over the video lecture ONLY. PowerPoints DO NOT cover the details needed for the Unit exam Each student is responsible for READING the TEXTBOOK for details to answer the UNIT OBJECTIVES Unit Objectives are your study guide (not this PowerPoint) Test questions cover the details of UNIT OBJECTIVES found only in your Textbook! 1  6/26/2024 Chapter 3 Objectives 1. Define quality assessment error analysis, phases of testing and process improvement. 2. List various ways a quality assessment program monitors laboratory testing. 3. Define quality control and describe how quality control, quality assurance, and process improvement relate to each other. 4. Define “standard” and describe how standards and calibrators are used in the laboratory. 5. List two types of errors that occur in analytical testing. 6. Define accuracy, precision; specificity and sensitivity 7. Describe how accuracy and precision relate to each other. 8. Define reference value and describe how a reference value is established. 9. Explain and demonstrate the calculation of the mean, the median, and the mode. 10. Explain and demonstrate the calculation of a standard deviation using the formula for standard deviation. 11. Describe the use of standard deviations in the assessment of laboratory testing. 12. Define “control sera” and analyte. 13. Describe how a control serum is used in the laboratory to provide quality control in laboratory testing. Chapter 3 Objectives (con’t) 14. Define Levy Jennings charts and describe how they are used to monitor testing. 15. Define coefficient of variation and demonstrate how it is used in the laboratory. 16. Define trend and shift; Interpret Westgard’s rules. 17. Explain and demonstrate how Westgard’s rules are used with Levy Jennings charts to determine if a test result may be reported to the physician. 18. Define the following key terms: a. Aliquot h. Out of control b. Serial dilutions i. PPV c. Percent j. POCT d. Proficiency testing k. Waived tests e. Osmolarity l. Moderately complex tests f. Calibration m. High complex tests g. PV 2  6/26/2024 Introduction Quality can be defined as worth of services Quality Assurance or Assessment is defined as: “Comprehensive set of policies, procedures and practices that ensure the laboratories’ results are reliable”. It includes record keeping, calibration and maintenance of equipment, quality control, proficiency testing and training. Satisfaction of the needs and expectations of the users of a service QA is in place from the time the patient checks into the lab until he/she receives the bill Quality Assessment Factors Non Analytical Factors in Quality Assessment 1. Qualified Personnel 2. Established laboratory policies 3. Laboratory Procedure manual 4. Test requisitioning 5. Patient identification, specimen procurement and labeling 6. Proper procedure for specimen collection and storage 7. Specimen transportation and processing 8. Preventive maintenance of equipment 9. Appropriate methodology Analytical Factors in Quality Assessment 1. Application of statistical analysis of results in quality assessment protocols 2. Instrumental applications of statistics to measurements 3  6/26/2024 Quality Assessment and Quality Control Quality results from analyses is critical Used by physician to discover the existence of disease and to follow progress Quality assessment (QA) ensures reliability of results. Must maintain preanalytical quality control (QC) in order to report valid laboratory results. Quality Assessment and Quality Control Quality Control is one part of Quality Assessment Quality Control is what runs the laboratory… QC let’s the tech know when results can be reported out and when results must be held Which is worse No results Wrong results QC is a way of life and must be understood 4  6/26/2024 Quality Assessment and Quality Control Functions of Quantitative Quality Control Program 1. Provides a guide to functioning of equipment, reagents, and individual technique. 2. Confirms the accuracy of testing when compared with reference values. 3. Detects an increase in the frequency of both high and low minimally acceptable values (dispersion). 4. Detects any progressive drift of values to one side of the average value for at least 3 days (trends). 5. Demonstrates an abrupt shift or change from the established average value for 3 days in a row (shift). Quality Assessment Responsibility of the clinical laboratory to ensure that results reported are reliable Clinical laboratory must give the physician an estimate of what constitutes “normal” values Quality Assessment (QA) and Total Quality Improvement (TQI) are driven by pressure to contain the cost of good service This is referred to as PROCESS IMPROVEMENT 5  6/26/2024 Standards Set by TJC The Joint Commission on Accreditation of Healthcare Organizations A private organization that inspects hospitals QA programs require monitoring and evaluation of quality and appropriateness of services to patients TJC has published a 10-step monitoring process for QA program 10-Step Monitoring Process 1. Assign responsibility for QA 2. Define the scope of patient care 3. Identify important aspects of care 4. Construct indicators 5. Define thresholds for evaluation 6. Collect and organize data 7. Evaluate data 8. Develop corrective action plan 9. Assess actions; document improvement 10. Communicate relevant information 6  6/26/2024 Components of QA Programs Documentation of ongoing QA program required since CLIA Commitment – Top Priority By Lab Persons working in lab must be totally committed to the concept of QA Facilities and Resources Layout of lab is important Vital lab is conducive to good performance and safe environment Technical Competence Essential to the quality of the lab QA is crucial in maintaining high level of performance by personnel CLIA requires certain levels of education for the types of tests personnel performs Ongoing review of competence Daily supervisory review of work Personnel must be trained to Perform analyses Perform quality control activities Maintain instruments Keep accurate and systematic records of reagents and controls; equipment maintenance; and patient and analytical data 7  6/26/2024 Quality Assessment Procedures Test requesting Patient identification Specimen procurement and labeling Specimen transportation and processing Analytic test procedures Turnaround times Accuracy of the final result Complete documentation of all procedures involved in obtaining the final analytical result must be maintained and monitored Quality Assessment Requirements Request form must be completed by the physician directing patient care Time of receipt of the specimen Necessary for monitoring turnaround times Request form must be clean and legible Information on accompanying specimen container must match request form Labs can accept only properly labeled specimens, if not the specimen is rejected 8  6/26/2024 Quality Assessment Requirements All containers must be labeled by the person doing the collection To make sure that the specimen has been collected from the patient whose ID is on the label Specimens must be transported to the lab in a safe, timely, and efficient manner A central receiving and processing area is set aside to record incoming specimens Specimens must be checked for acceptability Hemolysis or lipemia; specimen handling; amount Procedure Manuals Manuals are found in every department of the lab Outline of every procedure that department performs Procedures must be written in a certain format that are set by NCCLS (National Committee for Clinical Laboratory Standards) Manuals on all sorts of situations are found in the lab and all over the hospital where they are needed, manuals such as: Safety Personnel matters Specimen collection and handling 9  6/26/2024 Problem-Solving Mechanisms CLIA mandates documentation of any problem or situation that might affect the outcome of a test result All incidents must be documented in writing and must include Changes proposed, implementation and follow-up monitoring There must be a reasonable attempt to correct problems and situations All steps in the process must be documented Continuous Quality Improvement Ongoing process of making certain that correct laboratory results are reported out for the right patient in a timely manner and cost QA indicators are evaluated as part of the CQI Each laboratory will set its own indicators, depending on the goals of the laboratory Computer systems can assist in CQI by collecting data on tests, patients, turn-around-times and etc. 10  6/26/2024 CAP Quality Assessment Programs CAP (College of American Pathologists) provides assistance to the clinical laboratory in organizing and managing CAP QA programs All labs must have a QA program with Proper documentation defining the goals of the program Procedures necessary to achieve goals Specific records showing that procedures have been carried out CAP Quality Assessment Programs Comprehensive QA program must be designed to monitor and evaluate the ongoing and overall quality of the total testing process Preanalytic Analytic Postanalytic Must evaluate the effectiveness of policies and procedures Identify and correct problems Assure accurate, reliable, and prompt reporting Assure adequacy and competency of the staff 11  6/26/2024 Components of a CAP QA Program Patient test management Procedure manuals Quality control assessment Proficiency testing Comparison of test results Relationship of patient information to test results Personnel assessment Communications Diagnostic Value of Results Reported Diagnostic usefulness of a test and procedures are assessed by statistical evaluations Descriptors used: Reliability of procedure Accuracy and precision Variance Sensitivity Specificity Predictive value 12  6/26/2024 Accuracy verses Precision Accuracy Closeness of the result obtained to the true or actual value Precision The ability to get the same value in subsequent tests on the same sample Reproducibility It is possible to have great precision without accuracy Accuracy Aided by use of Properly standardized procedures Statistically valid comparisons of new methods with established reference methods Samples of known values (controls) Participation in proficiency testing programs 13  6/26/2024 Precision Can be ensured by: Proper inclusion of standards Reference samples Control solutions Statistically valid replicate determinations of a single sample Duplicate determinations of sufficient numbers of unknown samples Blind samples Accuracy vs Precision 14  6/26/2024 Reproducibility – Precision Can be expressed as Standard deviations (SD) Coefficient of variation (CV) Coefficient of Variation Used to compare the SD’s of two samples SD’s cannot be compared directly without using the mean (average) CV’s can be used to compare One day’s work with that of a similar day Compare test results from lab to the same type of test results from another lab CV formula: CV% = SD x 100 Mean 15  6/26/2024 Sources of Variance or Error It is impossible to obtain exactly the same result each time a test is performed Described as: Variance (or error) of a procedure Factors include: Limitations of the procedure itself Limitation related to the sampling mechanism Sampling Factors One of major difficulties in guaranteeing reliable results involves the sampling procedure A very small amount of blood it taken (5ml from 6 L) 1/1000 the of the total blood volume Time of day – circadian rhythm Patient’s position – lying or standing Patient’s physical activity – bed, ambulatory, physically active Interval of eating - fasting Time interval from blood collection to processing in lab Storage conditions 16  6/26/2024 Procedural Factors Other sources of variance of testing involves Aging of chemicals or reagents Personal bias of techs Laboratory bias due to Environment Methods apparatus Changes in instruments Changes in personnel Sensitivity and Specificity of a Test Both sensitivity and specificity are desirable characteristics for a test In different clinical situations, one is preferred over the other To determine sensitivity and specificity, four entities are needed Tests positive Tests negative Disease present (positive) Disease absent (negative) 17  6/26/2024 Sensitivity and Specificity of a Test TRUE POSITIVE Subjects who have a positive test result and also have the disease in question TRUE NEGATIVES Subjects who have a negative test result and who do not have the disease in question FALSE POSITIVE Subjects who have a positive test result and do not have the disease in question FALSE NEGATIVES Subjects who have a negative test result and do have the disease in question Sensitivity The sensitivity of a test is defined as the proportion of cases with a specific disease that give a positive test results! Those results that are positive are truly positive!!! Strep screens for strep throat – if the test says positive – the physician treats If strep screen reads negative then must do further testing 18  6/26/2024 Specificity The specificity of a test is defined as the proportion of cases with absence of the specific disease gives a negative test result Those test that are negative are truly negative!!! Syphilis screening – if test says you do not have syphilis then you do not!! If says you do, must do further testing Reference Values Definition of “Normal” or “Reference Values” Before physician’s can determine whether a patient is diseased they must have an idea of what is “Normal” It is the responsibility of the clinical laboratory to provide the physician with this information 19  6/26/2024 Reference Values Many factors enter into this determination Age Sex Race Geographic location Ethnic, cultural, and economic characteristics Internal analytic methods and practices Biometrics Science of statistics applied to biological observations Selection of a group on which to base “normals” For the basis of our discussion we are going to simplify Normals are based on patient population A well population from your patients Sometimes hard to get 20  6/26/2024 Reference Range Statistics Statistically, the reference range for a particular measurement is related to a bell-shaped curve referred to as Gaussian Curve or Distribution A statistically valid series of individuals who are thought to represent a normal healthy group from the patient population being served are measured and averaged Statistical Definitions Mean = X and means average of a set of numbers. The sum divided by the number of values Median = the middle value of a body of data. Variables are arranged in order of increasing magnitude, the median is that variable which falls into halfway between the highest and the lowest. 14, 15, 18, 18, 21, 26, 28, 29, 30 The median is 21 Mode = the value that occurs most commonly in the mass of data In the example above the mode is 18 21  6/26/2024 Standard Deviation Is the square root of the variance of the values in any one observation or in a series of results Standard Deviation or SD = the distribution around the average for a particular group 1 SD says 68% of all values in a measured group will fall into this area 2SD says 95% of all values in a measured group will fall into this area 3SD says 99.73% of all values in a measured group will fall into this area Developing a Reference Range In determining reference values for a particular measurement, a statistically valid series of people are chosen and assumed to represent a healthy population 1. 30 healthy people are tested 2. Test results are averaged to create a mean 3. Calculate a Standard Deviation Standard deviations are figured by the following formula: SD = ∑ (x1 – x)2 N-1 4. Calculate both the + and - 2 SD 5. Using the -2SD to +2SD calculations, develop the reference range The Laboratory always uses 2 Standard Deviations (2 SD’s) 22  6/26/2024 Reference Range Standard Deviation Standard deviations are figured by the following formula: SD = ∑ (x1 – x)2 N-1 X = each individual value X = mean S = Standard Deviation ∑ = Sum of N = number of tests performed = square root Reference Range Test Value (mg/dL) Mean X Deviation from Deviation squared X (874/10 = * 87.4 = 87) Mean (X – X) (X-X)2 * Must round # to original significant value 82 87 -5 25 85 87 -2 4 90 87 3 9 86 87 -1 1 91 87 4 16 90 87 3 9 81 87 -6 36 86 87 -1 1 94 87 7 49 89 87 2 4 Sum 874 Sum 154 23  6/26/2024 Calculations Mean X = sum of X n SD = 154 9 X = 874 = * 87.4 10 *Must round # to match original significant value SD = 17.1 X = 87 SD = * 4.13 *Must round # to match original Standard Deviation significant value ∑ (X-X)2 SDsq=rt 4 SD = N-1 SD = 4 2 SD = 8 Confidence Intervals Keep in mind that we expect 1 out of 20 (5 out of 100) to be out of the 2 SD (either below or above) of the Gaussian curve This means that we expect 2.5 out of 100 to be above the 2 SD range and 2.5 out of 100 to be below the 2 SD range 2.5 out of 100 is the same as 1 out of 40 to be high and 1 out of 40 to below 24  6/26/2024 Confidence Intervals Therefore, 95% of normal patient’s values will fall in the 2 SD Range 95% confidence interval is also used to explain the day-to-day shifts in values for a particular analytic procedure If we expect 95% to be within “normal” limits that means we expect 5% to be normal and outside the 2 SD range 5 out of 100 is the same as 1 out of 20 Quality Control Programs “The laboratory must establish and follow written quality control procedures for monitoring and evaluating the quality of the analytical testing process of each method to assure the accuracy and reliability of patient test results and reports.” CLIA Results given to physician MUST be clinically correct 25  6/26/2024 Quality Control Programs To ensure clinically correct results, laboratories use the following: 1. Standards Highly purified substance of a known composition 2. Control samples Specimen that is similar to the patient's blood with a known concentration of constituent. 3. Proficiency testing programs This is a program where samples are sent to a group of laboratories for analysis. Results are then compared with the other labs. This testing process is included as a component of the quality assurance program. Quality Control Program How Control specimens are used in the lab: 1. Similar in composition to the unknown specimen 2. Has a known concentration 3. Is included in every batch or run 4. It must be treated in the exact same way as the patient specimen and carried through the entire test procedure It is affected by any or all variables that affect the unknown (patient) specimen 5. The results must fall within the set QC rules to report patient results. 26  6/26/2024 Quality Control Program In controlling reliability of the laboratory results, the objective is to: Reject results when there is evidence that more than the permitted amount of error has occurred. Several ways of doing this: Run a control with every batch ( a collection of any number of specimens to be analyzed) or run Standard solutions Control specimens Duplicates Controls Chemistry controls are purchased and the same lot number is used for about a year. When the lot number is scheduled to expire New control data must be collected for about a month prior to expiration date A new mean and 2 SD range must be established for the new controls Controls are reconstituted with water, mixed, and allowed to come completely into solution Date of reconstitution, date of expiration, and initials of the tech who reconstituted must be on the control bottle. 27  6/26/2024 Controls Controls must be run at room temperature Controls must be kept closed and in the refrigerator between runs Collection of data Must mimic all conditions found when actually running the controls during the year When controls are new and old When instrument is clean and dirty By each and every tech who runs the instrument Must be run over a period of time Must be at least 30 but the larger the group of data the better the mean and range will be Controls The tech is still using the “old” lot number of control to determine if patient’s results can be reported while collecting data for “new” lot number. There are three control values for each test performed in the lab to make sure all possible areas of patient’s results are covered Abnormal High Normal Abnormal Low Instrumentation in the lab today collects data and figures out the mean and 2SD range 28  6/26/2024 Controls Once the data is collected a 2 SD range is established From this data a Levy-Jennings Chart is created The data used to create the chart is NOT plotted on the chart WHAT IS CHARTED ON THE LEVY-JENNINGS CHART??? ALL CONTROL RUNS NOT PATIENT RESULTS (PATIENTS ARE UNKNOWNS) Levy-Jennings Chart Gaussian Curve On It’s Side Is a Levy-Jennings Chart +2SD +1SD Mean -1SD -2SD 1 2 3 4 5 6 7 8 9 10 11 12 Each time control is run 29  6/26/2024 Levy-Jennings Chart Used to monitor controls for determining if patient results can be REPORTED or not Each time the controls are run the value is placed on the Levy- Jennings Chart The values are expected to fall up and down around the mean A value may fall outside the range 5% of the time or 1 out of 20 runs A value may fall above the mean 2.5% of the time or 1 out of 40 runs A value may fall below the mean 2.5% of the time or 1 out of 40 runs Shifts and Trends Shift FOUR consecutive control values above or below the mean. +2SD +1SD Mean -1SD -2SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Trends FOUR consecutive control values moving in one direction 30  6/26/2024 Control Program A quality control program is usually bought from a manufacturer At the end of each month the Levy-Jennings charts and control data is sent in to the company The company analyzes your data and compares it to other laboratories using the same methodology The company sends a report back to your lab giving your CV’s to determine your precision CV’s In order to calculate CV’s the formula is CV% = 1 SD X 100 Mean CV for 1st method CV for 2nd method CV = 4 X 100 CV = 2 X 100 87 87 CV = 0.045 X 100 CV = 0.023 X 100 CV = 4.5% CV = 2.2% The closer the CV value is to “1” the better the precision. Which one of the methods do you think is the best for reproducibility? CV’s are used for comparison’s To compare one method to another method To compare one lab to another lab 31  6/26/2024 Why are there QC Rules? To let us know when NOT to report out patients A minimum of two QC data points must be in to report. General QC Rules – DO NOT REPORT PATIENT RESULTS WHEN the following: There is a shift There is a trend One control is out of 2SD range in two consecutive runs Two out of three controls are out in one run Westgard’s Rules 12s Control rule that is commonly used with a Levey-Jennings chart when the control limits are set as the mean plus/minus 2s. In the original Westgard multirule QC procedure, this rule is used as a warning rule to trigger careful inspection of the control data by the following rejection rules. 32  6/26/2024 Westgard’s Rules 13s Control rule that is commonly used with a Levey-Jennings chart when the control limits are set as the mean plus 3s and the mean minus 3s. A run is rejected when a single control measurement exceeds the mean plus 3s or the mean minus 3s control limit. Westgard’s Rules 22s Reject when 2 consecutive control measurements exceed the same mean plus 2s or the same mean minus 2s control limit. 33  6/26/2024 Westgard’s Rules R4s Reject when 1 control measurement in a group exceeds the mean plus 2s and another exceeds the mean minus 2s. Westgard’s Rules 41s Reject when 4 consecutive control measurements exceed the same mean plus 1s or the same mean minus 1s control limit. 34  6/26/2024 Westgard Rules 10x Reject-10 consecutive points are above or below the mean Troubleshooting and Westgard Rules Identify the QC problem based on the previous control rules. NOTE: Not all of these rules need immediate attention. 1 2s rule is not so important as long as 3 levels of QC are run daily. 4 1s rule may indicate that it’s time to re-evaluate the mean. Determine the type of error: 1. Random Error(RE): Errors that affect reproducibility of a test system. This error affects values equally, both positive and negative. 1-3s or R-4s Poor reproducibility/Outliers 35  6/26/2024 Troubleshooting and Westgard Rules 2. Systematic error(SE): Errors that affect the accuracy or “closeness” to a true value. This error causes a bias, in a single direction only. 1-3s; 2-2s; 4-1s or 10x Shifts, Trends, or Calibration failure Then determine if it is: 1. Warning Rule: 1-2s 2. Rejection Rule: 1-3s; 2-2s; R4s, 4-1s, or 10x POTENTIAL ERROR SOURCES 1. RANDOM ERROR WORN PUMP TUBING VARYING BULB OUTPUT TECHNIQUE UNSTABLE REAGENT UNSTABLE ENVIRONMENT (TEMPERATURE, HUMIDITY,ELECTRICAL SUPPLY) 36  6/26/2024 POTENTIAL ERROR SOURCES 2. SYSTEMIC ERROR WORN PUMP TUBING DECREASED BULB OUTPUT IMPROPERLY CALIBRATED PIPETTOR NOTE: RE & SE CAN BE AFFECTED BY THE SAME PROBLEM GUIDELINES FOR TROUBLESHOOTING CHECK REFERENCE FLUIDS & LOT EXPIRATION DATES DATE OF LAST CALIBRATION? ARE THERE ONE OR MORE ANALYTES AFFECTED? IS THERE A PROBLEM IDENTIFIED BY THE WESTGARD RULE? IF SO IS IT SYSTMATIC OR RANDOM? EVALUATE THE POSSIBLE CAUSES TROUBLESHOOT BASED ON YOUR ANSWERS 37  6/26/2024 Quality Assessment Terms Used in Clinical Quality Assessment 1. Accuracy describes how close a test result is to the true value. 2. Calibration is the comparison of an instrument measurement or reading to a known physical constant. 3. Control (noun) represents a specimen that is similar in composition to the patient’s whole blood or plasma. 4. Precision describes how close the test results are to one another when repeated analyses of the same material are performed. Quality Assessment Terms Used in Clinical Quality Assessment 5. Standards are highly purified substances of a known composition. 6. Quality control is a process that monitors the accuracy and reproducibility of results through the use of control specimens. 7. Proficiency Testing is a means by which quality control between laboratories is maintained 38  6/26/2024 Postamble READ the TEXTBOOK for the details to answer the UNIT OBJECTIVES. USE THE UNIT OBJECTIVES AS A STUDY GUIDE All test questions come from detailed material found in the TEXTBOOK (Not this PowerPoint) and relate back to the Unit Objectives Postamble READ the TEXTBOOK for the details to answer the UNIT OBJECTIVES. USE THE UNIT OBJECTIVES AS A STUDY GUIDE All test questions come from detailed material found in the TEXTBOOK (Not this PowerPoint) and relate back to the Unit Objectives 39