Quality Control (QC) - Laboratory Statistics PDF

**Chapter 7** **Laboratory Statistic 2** **7-1. The Importance of Monitoring Patient Results: Reference Ranges and X-barM** **Reference ranges** provide an expected range of values in the general healthy population for a given analyte. (See the **image**, where the reference ranges are on the rig...

**Chapter 7** **Laboratory Statistic 2** **7-1. The Importance of Monitoring Patient Results: Reference Ranges and X-barM** **Reference ranges** provide an expected range of values in the general healthy population for a given analyte. (See the **image**, where the reference ranges are on the right side of the lab report.) Laboratories must verify each test method\'s reference ranges for their target population. (For example, a pediatric hospital would not report adult reference ranges as suggested by the method manufacturer.) - Reference ranges (normal values) should reflect the mean value in the population and a certain level of variation (usually 2 standard deviations). - 95% of all normal patients will fall within the reference range of an analyte. Values outside the reference ranges may indicate an abnormality in the patient or a problem with a given set of test results. (More on standard deviation is presented in the next topic section.) **The Importance of Monitoring Patient Results: XbarM and Reference Ranges** In addition to monitoring internal quality control results, patient results should be monitored. - As a general example: Should 20% of** patient results** suddenly begin to exceed a given reference range, there is most likely a testing error. X-barM is an example of a control tool (computer program algorithm) that continuously monitors analyzer performance and is capable of detecting any changes in real-time. - The X-barM process is also referred to as a system of moving averages (when referring to patient results averaged over time). - Laboratory analyzers that allow for X-barM QC calculations should contact the manufacturer\'s technical specialist for help in setting data collection targets. To summarize, systemic errors can be detected when patient results are monitored over time. **Mean** A QC program requires documentation of control results and routine assessment of data. It is important to understand the terminology and to properly interpret the statistical analyses of collected data. To begin, we will define basic terms used in the statistical analysis of data. - The **mean** is simply an average of data points. - Let\'s say that 100 glucose determinations are performed on a known control sample. - If we add all the results together, and then divide by the number of results, we obtain the mean. - Significant variations in the mean from day to day may indicate systematic errors. **Histograms** Plotting the results of glucose determinations on the X-Y axis will ideally produce a graph resembling the displayed **image**. - This graph is a **histogram**. The histogram represents all data points collected during a period of time. - A representation of all data points, when graphed, should ideally appear in the shape of a bell. - This is known as the **Gaussian curve,** more commonly referred to as a **bell-shaped curve**. **Standard Deviation** The graphic representation of data can then be divided into eight sections with the mean in the center of the X-axis. - Each of these sections is ***one* standard deviation or SD**. In the middle of these eight sections is a line that represents the mean. - The illustrated Gaussian curve shows a **normal distribution**, which indicates that most of the data points are close to the mean with very few of the data points being at one extreme or the other. **Acceptable Standard Deviation (SD)** - A small SD represents data where the results are very close in value to the mean. The larger the SD the more variance in the results. - Data points in a normal distribution are more likely to fall closer to the mean. In fact, 68% of all data points will be within ±1SD from the mean, 95% of all data points will be within [+] 2SD from the mean, and 99% of all data points will be within ±3SD. - Statisticians have determined that values no greater than plus or minus 2 SD represent measurements that are closer to the true value than those that fall in the area greater than ± 2SD. Thus, most QC programs require that corrective action be initiated for data points routinely outside of the ±2SD range. https://www.medialab.com/courses/imgs/1026-38595.jpg **Establishing Quality Control Limits** **Whether using assayed controls or unassayed controls, acceptable ranges for each lot of control materials will need to be calculated.** Sufficient data points must be collected to guarantee that the calculations are valid and provide assurance that patient test results are as accurate as possible. For the data to reflect future system performance, the measurements must be made under conditions representative of the normal variation encountered during daily testing. Possible variations in testing include: - Changing calibrator lots - Performing calibrations - Changing reagent lots - Changing bottles of control material - Changing operators - Data collected at different times of the day - Changes in sample probes or pipettes - Post-maintenance changes **Important:** Data should **not** be collected during abnormal or out-of-control conditions. All valid data points should be used in calculations or the control limits may become too restrictive. This would not reflect the actual variability of the test system. Including quality control data from afternoon and night shifts makes sure you cover all results and all employees in your quality control program. - **Quality control results that exceed [+] 2 standard deviations (SD) should [not] be deleted when you are [establishing] QC limits as this action can distort or skew the limits.** - However, if **more than** two data points appear to be significantly higher or lower than the rest of the data, this could be an indication of a problem. In this case, you should **resolve the problem and recollect the data**. Using control data from the previous month can help you predict what may occur this month, thus helping with calculations. Using target values suggested by the manufacturer provides an example of the likely values that will be encountered. **7-2. Levey-Jennings Control Charts** Daily documentation and evaluation of quality control are vital to the detection of errors.\ One of the most commonly used methods for documentation is the Levey-Jennings control chart (**L-J chart**). - In 1931, Dr. Walter Shewhart, a scientist at the Bell Telephone Laboratories, proposed the application of statistical-based control charts to monitor industrial manufacturing processes. - In 1950, S. Levey and E.R. Jennings applied Dr. Shewhart\'s control charts to the clinical laboratory **What is a Levey-Jennings Chart?** It is a graph on which analyte quality control data is plotted to provide a visual indication of whether a laboratory test method is working correctly. - The Levey-Jennings chart usually has the days of the month plotted on the X-axis and the control observations plotted on the Y-axis. - On the right is the **Gaussian **or \"bell-shaped\" curve turned on its side to show the correlation of the curve to the chart (ie, fewer data points should appear on the upper and lower extremities of the chart, since the \"bell\" is thinner farther from the mean). - By observing the data plotted in the L-J chart, we can determine if test results are in control and accurate, or if test results are not in control and consequently unacceptable. - **Use of the Westgard, will help establish an effective error-detecting scheme.**  **7-3. Westgard Rules** In 1981, Dr. James Westgard and his associates developed a multi-rule procedure for interpreting control data. - Since then, several sophisticated quality control schemes or analogs based on this multi-rule logic have evolved. - To show how the **Westgard Rules** may be applied in quality control, four of the *most* common rejection limits will be illustrated in the following pages. **Westgard Rule 1~3S~** **Westgard rule 1~3s~** states that: - If a control is greater than ± 3 standard deviations from the mean, it should be rejected and rerun. - This is because either a random error or a very large systematic error has occurred, as less than 1% of all test values exceed ± 3SD. - In the accompanying **image**, the control for Day 13 (noted by the arrow) is greater than +3SD from the mean. Consequently, the 1~3s~ rule applies and *the run is rejected*. Troubleshooting must be performed before further testing can be done. https://www.medialab.com/courses/imgs/1026-38573.jpg **Westgard Rule 2~2S~** **Westgard rule 2~2s ~**states that: - If two *consecutive* control measurements exceed the same mean by +2 SD or by -2 SD ( or, within a run, if two consecutive control values are outside the same 2 SD) *the run must be rejected*. - If this circumstance occurs, a systematic error is likely. - The **top chart** represents the day\'s \"normal\" control, while the** bottom chart **shows the day\'s \"elevated\" control. - The L-J plots on the 13th day for both the normal and elevated controls show greater than +2SD. Troubleshooting must be performed before testing can continue. - Had only one of the controls been greater than +2SD, the run would have been accepted as \"in control,\" but would have been rejected on the next QC run if the same control was again out +2SD.  **Westgard Rule 4~1s~** **Westgard Rule 4~1s ~**states that the run should be *rejected* if: - Four consecutive control measurements have exceeded the same mean plus 1 standard deviation or the same mean minus 1 standard deviation. **Westgard Rule R~4s~** The** Westgard Rule R~4s ~**applies to controls **within a run (and not between runs)**. - If two controls exceed 4SD (that is if one control exceeds +2SD and the other control \[or additional control, if more than 2 controls are tested\] exceeds -2SD), *the run should be rejected*. - This is the case on day 13, as seen on the normal and abnormal L-J QC **charts** on the right. **10x Rule** This rule applies: - If ten consecutive QC results for one level of control are on one side of the mean the mean has "shifted" and an error has occurred. - also if both levels of control have five consecutive results that are on the same side of the mean. **7-4. Trend and Shift** **Descriptive term: Trend** A **trend** shows a gradual, continuous movement of the data values in one direction away from the mean, over time. - A trend is indicative of a systematic error that is occurring gradually, such as with reagent deterioration, control deterioration, or a lens getting progressively dirtier. - In the **example** to the right, some type of systematic error began on day 6, and by day 10, we can say that a trend has occurred and results are out of control. - Be on the lookout for trends as they begin to take shape, and you may be able to avoid further quality control problems.  **Trend** **Descriptive term: Shift** A **shift,** on the other hand, describes a sudden change in the data mean that persists throughout further days of testing. - Shifts can be caused by any one of a number of reasons, including persistent instrument malfunction, loss of calibration, or improper reagents. - Shifts are often easier to recognize than trends, as the data may be more clearly suspect. - Again, shifts must be corrected as soon as possible, as patient data are most likely already invalid. - In the** example** to the right, the large jump on day 8 followed by persistently high control values means that a shift has occurred. **Shift** **7-5. Troubleshooting QC** Problems are inevitable in quality control (QC). Some problems affecting QC can be avoided simply by: - performing regularly scheduled cleaning, and - other maintenance of equipment. Now is also a good time to remind everyone that proper training (and competency testing) of laboratory staff is required. It does no good to troubleshoot QC scenarios if the staff is not performing the task(s) correctly. Other problems, though, happen suddenly and require immediate action to resolve an analysis that is \"out of control\". - Every clinical laboratory has its own protocol to be followed when controls are outside accepted limits. It is the responsibility of the medical laboratory professional to be familiar with this procedure and follow it explicitly. - The steps on the next pages are examples of some corrective procedures that a medical laboratory may use to restore [precision](https://www.medialab.com/courses/s_page.aspx?step=67&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) and [accuracy](https://www.medialab.com/courses/s_page.aspx?step=67&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39). These include: *1. Check your reagents and controls.* *2. Rerun the control that is out-of-range.* *3. Run the control using a new unopened bottle of control.* *4. Review the calibration of the test instrument.* *5. Call the test manufacturer for advice.* **7-6. Corrective Actions** ***1. Check your reagents and controls.*** - Verify that all lot numbers and expiration dates of reagents and controls used in the testing process are current. - Verify that the recommended storage conditions were followed. (Not only the material was placed in the proper storage location, but this location has been monitored for proper performance and temperature.) In addition: - New lots of QC material should always be run in parallel with the old lot number - before being used for analysis. This may also be referred to as a** [crossover](https://www.medialab.com/courses/s_page.aspx?step=68&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) study**. - ***Crossover-**A crossover study is performed by testing the existing QC lot (before the current QC lot runs out or expires) alongside or parallel to the new lot of QC material. This is done to establish new QC ranges. A defined number of runs must take place before calculating the new range for the new lot.* - This type of study is performed by testing the existing QC lot (before the current QC lot runs out or expires) alongside or parallel to the new lot of QC material. This is done to establish new QC ranges. A defined number of runs must take place before calculating the new range for the new lot. It is possible that the new lot number was placed in use and this is the reason for the out-of-control results. - Keep a check on water quality (as some QC material requires the addition of laboratory-grade water to the lyophilized material). Perhaps the lab water source has become contaminated due to poor maintenance. - Monitor glassware and pipettes for proper calibration checks. Perhaps the pipettor is broken and does not measure correctly. (Remember, these items may be used by the laboratory worker when preparing QC material.) - Also ask: Did the laboratory worker prepare the QC material properly? (Proper training? Training documentation? Competency checks performed?). ***2. Rerun the control that is out-of-range.*** - Random errors in sampling may be resolved by simply repeating the test using the same control and a fresh testing device. ***3. Run the control using a new unopened bottle of control.*** - Improper storage may have accelerated the deterioration of the original control or the testing material may have expired. (Improper storage might include leaving the QC material out on the benchtop for an entire day - when it was to be placed back into the refrigerator immediately after use.) ***4. Review the calibration of the test instrument.*** - What was the date of the last calibration? - Test instruments need to be calibrated according to the manufacturer\'s instructions or more frequently if necessary. - Federal requirements call for analytical tests to be recalibrated, or calibration verification performed, at least every six months to verify the [accuracy](https://www.medialab.com/courses/s_page.aspx?step=71&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) of the testing procedure. (Additional verification must be performed whenever there has been a major change/repair/software update to the analyzer or test system.) ***5. Call the test manufacturer for advice.*** - Manufacturers have additional information that may help resolve QC problems. - Patient test results cannot be reported if the QC problem cannot be resolved. - If the problem cannot be resolved, specimens should be stored properly for later testing or sent out for testing at another facility or reference laboratory, as per your laboratory\'s protocol for this situation. **Note:** Always document any corrective actions performed when troubleshooting suspect quality control results. The documentation must be reviewed by the supervisor; any nonconforming events must undergo review by the laboratory quality management team. **7-7. Verification of Performance Specifications for Nonwaived Testing** One last topic to discuss, and this will help to bring all of the previously discussed quality control terms and tools into focus - as they are utilized in the packages of validation and verification. (For an example of how this information may be represented, review any laboratory test method package insert, and you should find detailed information on how the test method was validated for use.) **Validation versus verification** The process of validation of test methods falls upon the initial manufacturer of the test system/method. Laboratory-developed tests may also be required under law to provide detailed documentation of new test validations. Once a new test has been validated, the verification process is required by the end user. Each laboratory must verify that the test system/method meets the manufacturer\'s specifications. The U.S. Clinical Laboratory Improvement Amendments (CLIA) require laboratory users of [nonwaived](https://www.medialab.com/courses/s_page.aspx?step=74&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39), unmodified, FDA-cleared or approved test systems to perform the following before reporting patient test results: - Demonstrate that the test system is capable of obtaining performance specifications comparable to those established by the manufacturer for the following performance characteristics: - [**Accuracy**](https://www.medialab.com/courses/s_page.aspx?step=74&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) - [**Precision**](https://www.medialab.com/courses/s_page.aspx?step=74&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) - **Reportable range** of test results for the test system - **Verify that the manufacturer\'s reference ranges **(normal values) are appropriate for the laboratory\'s patient population. **Verification for Nonwaived Testing** Laboratories that perform **[nonwaived](https://www.medialab.com/courses/s_page.aspx?step=75&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) testing** must verify that they can obtain performance specifications comparable to those established by the manufacturer. - ***Nonwaived-**The Centers for Medicare and Medicaid Services (CMS) regulates all laboratory testing (except research) performed on humans in the United States through the Clinical Laboratory Improvement Amendments (CLIA). CLIA-waived tests are simple laboratory examinations and procedures that employ methodologies that are considered by the U.S. Food and Drug Administration (FDA) to be (1) so simple and accurate as to render the likelihood of erroneous results negligible, or (2) the test poses no reasonable risk of harm to the patient if the test is performed incorrectly. If a test method is not labeled as waived, it is nonwaived and therefore subject to CLIA regulations.* - Top of Form - - Generally this can be accomplished by doing **split-sample comparison studies **(a technique where a single patient sample is split into two aliquots - one is tested using the primary method, while the other is sent to a lab that uses an appropriate comparison method) to estimate any inaccuracy or [bias](https://www.medialab.com/courses/s_page.aspx?step=75&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39), plus **[linearity](https://www.medialab.com/courses/s_page.aspx?step=75&currentassignment=52991188&currentassignmenthash=566ffb4151983229e3d8272d8b623918&oid=104592847&o=73edaf10f7be75b14404a340d7790e39) studies **to estimate imprecision and determine the **reportable range**. - ***Bias-**Systematic error or bias refers to deviations that are not due to chance alone. Rather, they may be due to the method.* - ***Linearity-**Linearity is the ability to provide laboratory test results that are directly proportional to the concentration of the quantity of whatever is being measured. This is graphed on the X and Y axis to reflect the degree of linearity.* - Top of Form - - The laboratory can do studies to determine its own **reference ranges**, or the laboratory director can document that the manufacturer\'s ranges are appropriate (after reviewing the demographics of the laboratory\'s target patient population). - **Note**: For those laboratories performing nonwaived tests that have been modified or developed in-house, additional verification studies are required. (Refer to CLIA requirements, often detailed within laboratory accreditation checklists.) **Summary** - This course has presented basic quality control ideas (terms, formulas, and a few basic statistical analysis methods) as they pertain to the examination phase of laboratory testing. - Each medical laboratory discipline (eg, hematology, chemistry, immunology, urinalysis, microbiology, and so forth) in each laboratory facility will define a QC Program to fit under the overarching Quality Management System. Review of statistics occurs during the short-term (as the patient tests are performed), and over the long-term (weekly, monthly, annually) as needed and as defined by the laboratory management and director. - **As a reminder:** Always refer to your individual facility\'s policies and procedures for specific process instructions.