Quality Assurance CHAPTER ONE PDF

Chapter 1 Introduction to Quality Assurance  Learning Objectives Upon completion of this chapter the student will be able to: Define quality assurance List important terms in quality assurance Discuss the basic principles of quality assurance Describe the different components of qual...

Chapter 1 Introduction to Quality Assurance  Learning Objectives Upon completion of this chapter the student will be able to: Define quality assurance List important terms in quality assurance Discuss the basic principles of quality assurance Describe the different components of quality assurance Describe the importance and necessity of quality assurance Classify types of diagnostic tests based on type of result obtained and method of analysis Describe Accuracy and precision Learning Objectives  Describe indicator values of diagnostic tests including sensitivity, specificity, test efficiency, predictive values.  Calculate the following diagnostic values: Sensitivity Specificity Overall Test Efficiency Predictive values  Use the diagnostic values to determine acceptability of test methods Chapter Outline  Definitionof important terms in quality assurance  Essential components of quality system  Aspects of quality assurance  Characteristics of quality assurance  Basic components of Quality assurance program  Benefits of Quality assurance  The purpose of health laboratory Chapter Outline  Errors in the clinical laboratories Clerical Errors Sampling errors Analytical errors  Types of Diagnostic tests  Accuracy and precision Chapter Outline  Indicators of values of diagnostic test Sensitivity Specificity Test Efficiency Predictive values What is Quality Assurance? QA: development and implementation of measures to assure accurate, precise and reliable laboratory service  What Is “Quality?” A measure of excellence or a state of being free from defects, deficiencies and significant variations. The ability of a product or service to satisfy stated or implied needs of a specific customer Quality is a measure of excellence or a state of being free from defects, deficiencies and significant variation The degree to which a set of inherent characteristics fulfills requirement. (as per ISO 9000:2005) Achieved by conforming to established requirements and standards. The totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.  Dimension of Quality fig HIGH Excellent Standard Fitness for Consumer Externally Internally purpose Absolute satisfaction Relative Relative Low Standard Low  QA QA: is an ongoing systematic comprehensive evaluation of health care services and the impact of those services on health care services. All activities under taken to predate and prevent poor quality. QA programme: is the sum of all activities and procedures undertaken by medical laboratories to improve the quality and clinical usefulness of laboratory test results. How do we perform quality assurance in laboratory? Quality Assurance, CLIA, and Your Lab 1.Assess the effectiveness of the lab's policies and procedures. 2.Identify and correct problems. 3.Assure the accurate, reliable, and prompt reporting of test results. 4.Assure the adequacy and competency of the staff. Principle of QA….. Customer focus Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision making Mutually beneficial supplier relationship Key terms related to QA Quality improvement Total quality management/continuous quality improvement Quality control Quality circle Laboratory quality Laboratory quality is can be defined as, accuracy of result, reliability of result, timeliness of result ( WHO Definition) The lab test result is accurate as possible  Consequences of inaccurate result  Unnecessary treatment  RX complication  Failure to provide proper RX  Delay in correct DX  Additional & unnecessary DX QUALITY MANAGEMENT SYSTEM QMS: Coordinated activities to direct and control an organization with regard to quality (ISO & CLSI) Quality Assurance vs Quality Control QA: is planned and systematic activities to assure quality lab results QC: is a procedure, samples of control material and rules  Designed to assess for analytic errors  Part of Quality Assurance in the Analytic Stage QA & QC QA QC The planned and The observation and systematic activities activities to fulfil implemented in a quality requirement for quality. system, so that quality Mechanism used, such requirements for a as checks or test that product or service will be performed to ensure that fulfilled. retequirement are me  Fit for purpose  Right first time QA & QC Cont… QA QC Focus on Prevention of Focus on identifying error error Product(analysis) Process oriented oriented Pro active(prention from Reactive( check ) arising error) Implement at the final Implement at design stage stage Identify error before test Prevent error from result reported beginning to the end of process Definitions of Terms Lab process: interrelated activity from input to output ( how it happens) Pre-analytical: before testing Analytical: testing phase Post-analytical: after testing Clerical Errors: mistakes in writing results Sampling errors: mistakes in the specimen Accuracy : closeness of result to true result Precision : closeness of replicated results of the same sample to each other Definition of a Lab Quality management System The organizational structure, responsibilities, processes, procedures, and resources for implementing quality management of the laboratory or testing site In other words… all activities which contribute to quality of tests, directly or indirectly Why is the Quality System Important to Patient Sample Testing? Ensures that quality is the foundation of everything we do Sets the standard for level of quality Meets/exceeds customer expectations Provides means to prevent, detect and correct problems Becomes the core of a monitoring, evaluation, & improvement system Reduces costs Even the simplest Rapid Test is not foolproof Without a Laboratory Quality System - too many mistakes can make analysis very costly; due to costs of analysis expenses caused by wrong decisions, or repeating analysis of samples investigation of problems revision of procedures loss of good reputation Prevention is Better than cure! ‘It costs less to prevent a problem than it does to correct it’ A formal quality system in the laboratory should prevent mistakes by means of: quality assurance measures quality control of the analytical results thorough documentation of the system efficient maintenance of records regular audits of all aspects of the system QA Systems Quality Assurance measures apply to the laboratory analytical work overall, which includes; identifying the person having the overall responsibility for quality having laboratory equipment calibrated to recognised standards using reference materials joining proficiency testing schemes with other laboratories doing similar tests  % Fat 3.30 QC systems 3.25 % fat Quality control measures apply to 3.20 each analytical 3.15 test in the laboratory by use of: 3.10 14/01/02 24/01/02 03/02/02 13/02/02 23/02/02 05/03/02 15/03/02 25/03/02 04/04/02 14/04/02 24/04/02 date Control chart reagent blanks; verified standard blind samples solutions; replicate analyses; check samples and control charts (from both within the lab and from outside); Essential of a Quality System Organizati Personne Equipme on l nt Process Purchasi Control Informatio Quality ng & Control & n Specimen Manageme Inventory Managemen nt t Documen Occurrenc e Assessm ts & Manageme ent Records nt Process Custome Facilities Improveme r Service & Safety nt  Organization Quality Policy Sufficient & Standards Resources Clearly Defined A Culture Roles Committed & Accountability to Quality  Personnel Human Resource Hiring Retention Planning Performance Training Supervision Management Quality System Equipment Installation & Selection Acquisition Initial Calibration Maintenance, Troubleshooting Disposition Service & Repair Quality System Purchasing and Inventory Procurement Receiving Storage Inventory Record Management Keeping Quality System Process Control Standard Specimen Operating Management Procedures Quality Control Quality System Documents and Records Standardized Document Document Forms Approval Distribution Document Document Storage/Retrieval Destruction Quality System: Information Management Information Data Collection Flow & Management Patient Privacy & Computer Confidentiality Skills Quality System: Occurrence Management Written Corrective Procedures for Actions Addressing Errors Occurrence Occurrence Records Reporting Quality System Essential: Assessment External Quality Assessment Internal Audit Improvement or Self Measures Evaluation Quality System Process Improvement On-Going Improvement Data Measures Collection Quality System: Service and Satisfaction Monitoring Process Customer Improvement Satisfaction Rewards Quality System Facilities and Safety Testing and Storage Safety Practice Areas Safety Procedures & Records Total Quality Management TQM addresses all these areas of laboratory practice Lab service and resources Purpose of Health Laboratory Provide patient laboratory results  Diagnostic  Prognostic  Monitor treatment  Monitor disease outbreaks Characteristics and Aspects of Quality Assurance Three phases affect  Useful patient results Aspects of each phase are shown on next diagram. The Quality Assurance Cycle Patient/Client Prep Sample Collection Personnel Competency Reporting Test Evaluations Data and Lab Management Safety Customer Service Sample Receipt and Accessioning Record Keeping Quality Control Sample Transport Testing Basic Components of QA Internal quality assessment (IQA) External quality assessment (EQA) Standardization of processes and procedures (pre-analytic, analytic and post-analytic phases) Management and 0rganization Benefits of Quality Assurance Helps physicians, patients and clients Creates good reputation Motivates staff Is cost-effective Prevents complaints Builds trust Three Stages of Analysis Pre-analytic Analytic Post-analytic Errors in each stage should be prevented Pre-Analytic Errors Process Potential Errors Test Ordering Wrong test for patient, not legible, wrong patient, delay Specimen Requirements not met including patient Collection ID, wrong tube, volume, poor sample or wrong time. Specimen Wrong transport conditions. Handling Pre-Analytic Specimen Collection Errors Wrong:  Patient ID  Anticoagulant  Volume  Process Haemolysis IV contamination Prolonged tourniquet  Preventing and Detecting Errors – Before Testing Check storage and room temperature Select an appropriate testing workspace Check inventory and expiration dates Review testing procedures Record pertinent information, and label test device Collect appropriate specimen  Analytic Errors During testing phase errors may occur  Auto-monitoring  Error codes  Quality control result rejection signals Data Error Flags: AMS Autolab Code Causes of Error CHECK Blank absorbance reading, QC sample result or %CV outside of limits L Higher than upper linearity limit Quality Assurance vs. Quality Control Quality Assurance Quality Control Activities to ensure process Activities to evaluate a are adequate for a system to product or work result Definition achieve its objectives Establish standard Analyze known QC procedures for sample sample to determine if a collection test is valid Examples Define criteria for Decide if a sample is acceptable samples acceptable for testing  Quality Control Material Assess Analytic phase  Acceptance versus rejection of patient results  2 or 3 control materials  It is a process or system for monitoring the quality of laboratory testing accuracy and precision of results Quality Control Tracking QC Rules Guide acceptance or rejection of patient results based on QC results  95% confidence limits  Westgard multi-rules Preventing and Detecting Errors – During Testing Perform and review Quality Control (QC) Follow safety precautions Conduct test according to written procedures Correctly interpret test results Post-Analytic Errors Examples include: Clerical errors(Transcription errors Report illegible Report sent to the wrong location Information system not maintained Wrong reference ranges Not reporting in a timely manner Not maintaining confidentiality Preventing and Detecting Errors – After testing Re-check patient/client identifier Write legibly Clean up and dispose of contaminated waste Package EQA specimens for re-testing, if needed EQA = external quality assessment Example Exercise: When Patient Result is Outside the Expected Result A previously confirmed HIV-positive patient is being re-tested by HIV Rapid Test and the test results are negative. What could be the problem? Example Exercise: When Outside the Expected Result In order to troubleshoot why your patient result is outside the expected result, it is important to take into consideration all factors that effect the test result, from the time the test is ordered Let’s go through these: Pre-analytic Analytic Post-analytic Quality of collection Pipetting technique Timely resulting Age of specimen Instrument problem Communication of Specimen transport Clerical errors results to patient file Specimen Analytical errors Record keeping acceptability  Types of Diagnostic Test Results Qualitative: positive or negative (reactive or non reactive)  Rapid HIV  others Semi-quantitative: estimated  Urinalysis biochemical results  ASO Titer  Others  Types of Diagnostic Test Results Quantitative: numerical amount with SI unit  WBC count  CD4 count  Hemoglobin g/dL  Glucose mg/dL  Others Accuracy and Precision Quality Control testing of samples and rules are used to monitor both the precision and the accuracy of the assay in order to provide reliable results. Accuracy The closeness of the measured result to the true value Possible causes of inaccuracy:  Lack of technologist skill Dirty cuvette for analysis  Poor mixing  Evaporation of controls  Evaporation of standards  Reagent deterioration  Poor pipetting eg. using the wrong type of pipette Precision Reproducibility or closeness of results to each other Reliability of Measurement Ability to maintain both precision and accuracy Accuracy: refers to the closeness of the measured result to the true value Precision: refers to the reproducibility of the result when repeatedly measured Inaccurate and Imprecise Precision Calculations Precision is determined by % coefficient of variation (%CV) % CV = (standard deviation x 100%)/ mean %CV relates to % of Precision error Acceptable %CV < 5% (method dependent) Practice calculating % CV given the following: Study was performed for 50 uL pipette with the following results: Volumes obtained from gravimetric testing of distilled water. Mean of volumes = 49.2 uL standard deviation = 3 uL. Accuracy Calculations Accuracy can be determined from bias:  Calculating the difference between the measured and true value  |True value-mean| / True Value x 100%  % Accuracy error should be < 5%  Practice calculating % accuracy error  Study was performed for 50 uL pipette with the following results: Volumes obtained from gravimetric testing of distilled water. Mean of volumes = 49.2 uL standard deviation = 3 uL. Indicators of Diagnostic Tests Clinicians rely on the laboratory to provide accurate test results  Aidin diagnosis of disease  Determine the patient’s prognosis  Determine effectiveness of treatment  Determine relative risk of contracting disease Indicators of Diagnostic Tests There are several statistical assessments that can be of value in evaluating the diagnostic usefulness a test This process relates the patient’s test results with the presence or absence of disease The ability of the screening test to differentiate between those who are disease free from those who are affected is called the test validity. Indicators of Diagnostic Tests and Medical Usefulness All methods have an inherent amount of error present that will affect test results No method is able to detect all persons with disease accurately No method is able to detect all persons without disease accurately Four Possible Outcomes to Test Results and Disease Diagnosis True positive (TP)  A positive test result for patients who have the disease True negative (TN)  A negative test result for patients who do not have the disease False positive (FP)  A positive test result for patients who do not have the disease False negative (FN)  A negative test result for patients who do have the disease Results of Disease Screening Testing: Possible Outcome A = true positive (TP) b = false-positive (FP) C = false negative (FN) D + true negative (TN) Indicators of Diagnostic Values of Test Definitions Test Sensitivity = Diagnostic Sensitivity  Percentage of patients who have a disease that test positive (true positive) Test Specificity = Diagnostic Specificity  Percentage of patients who do not have disease that test negative (true negative) No test is 100% sensitive and 100% specific Test Efficiency is the relationship between specificity and sensitivity of a test.  Definition of Analytical Sensitivity Analysis able to detect small amounts of the analyte Before a test can be sensitive for disease detection it must have analytical sensitivity The probability of a positive test result given the presence of disease How good is the test at detecting infection in those who have the disease? A sensitive test will rarely miss people who have the disease (few false negatives). Sensitivity The sensitivity of a test in the ability of the test to identify correctly affected individuals Proportion of persons testing positive among affected individuals Affected persons (Positive by gold standard) Persons testing positive Persons testing negative (True positives) (False negatives) Sensitivity = True positives / Affected persons Estimate the 95% confidence interval Estimating the sensitivity of a test Identify affected individuals with a gold standard Obtain a wide panel of samples that are representative of the population of affected individuals  Recent and old cases  Severe and mild cases  Various ages and sexes Test the affected individuals Estimate the proportion of affected individuals that are positive with the new test methods Estimating the sensitivity of a rapid test for leishmaniasis Identify persons with leishmaniasis with a gold standard  Parasitologically proven infection Obtain a wide panel of samples that are representative of the population of individuals with leishmaniasis  Recent and old cases  Severe and asymptomatic cases  Various ages and sexes Test the persons with leishmaniasis Estimate the proportion of persons with  Sensitivity and Specificity DISEASE Present Absent True False Positive Positive Positive (TP) (FP) TEST False True Negative Negative Negative (FN) (TN) Sensitivity = TP/TP+FN Specificity = TN/TN+FP Sensitivity of a rapid test for leishmaniasis Patients with leishmaniasis True positive 148 Rapid test False negative 2 150 Sensitivity = 148 / (150) = 98% 95% confidence interval: 94%-99% What factors influence the sensitivity of a test? Characteristics of the affected persons?  YES: Antigenic characteristics of the pathogen in the area (e.g., if the test was not prepared with antigens reflecting the population of pathogens in the area, it will not pick up infected persons in the area) Characteristics of the non-affected persons?  NO: The sensitivity is estimated on a population of affected persons Prevalence of the disease?  NO: The sensitivity is estimated on a population of affected persons Definition of Analytical Specificity Analysis able to distinguish one analyte apart from similar substances Before a test can be specific for a disease it must have analytical specificity The probability of a negative test result given the absence of disease. How good is the test at calling uninfected people negative? A specific test will rarely misclassify people without the disease as infected (few false positives).  Specificity The specificity of a test in the ability of the test to identify correctly non-affected individuals Proportion of person testing negative among non affected individuals Non-affected persons (Negative by gold standard) Persons testing negative Persons testing positive (True negatives) (False positives) Specificity = True negatives / Non-affected persons Estimate the 95% confidence interval Laboratory Training for FieldEEpidemiologists P I D E M I C A L E R T A N D R E S P O N S E  Estimating the specificity of a test Identify non affected individuals – Negative with a gold standard – Unlikely to be infected Obtain a wide panel of samples that are representative of the population of non-affected individuals – Diverse unaffected population: Difficult to find. Ideally, those individuals that would need to be tested but not infected Test the non-affected individuals Estimate the proportion of non-affected individuals that are negative with the test Laboratory Training for FieldEEpidemiologists P I D E M I C A L E R T A N D R E S P O N S E  Estimating the specificity of a rapid test for leishmaniasis Identify persons without leishmaniasis – Persons without sign and symptoms of the infection – Persons at low risk of infection, negative with gold standard Obtain a wide panel of samples that are representative of the population of individuals without leishmaniasis – Persons from neighbouring villages having similar characteristics but with no transmission and no infections Test the persons without leishmaniasis Estimate the proportion of persons without leishmaniasis that are negative with the rapid test Laboratory Training for FieldEEpidemiologists P I D E M I C A L E R T A N D R E S P O N S E  Sensitivity and Specificity DISEASE Present Absent True False Positive Positive Positive (TP) (FP) TEST False True Negative Negative Negative (FN) (TN) Sensitivity = TP/TP+FN Specificity = TN/TN+FP  Specificity of a rapid test for leishmaniasis Individuals without leishmaniasis False positive 12 Test True negative 188 200 Specificity = 188 / 200 = 94% 95% confidence interval: 90%-96% Laboratory Training for FieldEEpidemiologists P I D E M I C A L E R T A N D R E S P O N S E  What factors influence the specificity of a test? Characteristics of the affected persons?  NO: The specificity is estimated on a population of non affected Characteristics of the non-affected persons?  YES: The diversity of antibodies to various other antigens in the population may affect cross reactivity (e.g., If malaria is endemic, polyclonal hyper gammaglobulinemia may increase the proportion of false positives) Prevalence of the disease?  NO: The specificity is estimated on a population of non affected Specificity is an INTRINSIC characteristic of the test Laboratory Training for FieldEEpidemiologists P I D E M I C A L E R T A N D R E S P O N S E Identifying the cut-off to use with a test on the basis of panel analysis: Ideal case 25 20 Cut-off Number of tests 15 Sick 10 Well 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Possible values of the test Identifying the cut-off to use with a test on the basis of panel analysis: Real case 25 Cut-off 20 False False Number of tests negatives positives 15 Sick True 10 negative Well s True 5 positive s 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Possible values of the test To whom sensitivity and specificity matters most? Look at denominators! – Panels of affected individuals – Panels of negative individuals To laboratory specialists!.. Intrinsic characteristics of a test – Sensitivity – Specificity Performance of a test in a population – Predictive value of a positive test – Predictive value of a negative test  How is the test doing in a real population? The test is now used in a real population This population is made of – Affected individuals – Non-affected individuals The proportion of affected individuals is the prevalence Status of persons Affected Non-affected Positive True + False + A+B Test Negative False - True - C+D A+C B+D A+C+B+D The predictive value of a positive test is the probability that an individual testing positive is truly affected Proportion of affected persons among those testing positive Predictive value of a positive test Persons testing positive (Positive by test) Persons affected Persons not affected (True positives) (False positives) Predictive value of a positive test = True positives / Persons testing positive Estimate the 95% confidence interval  Predictive Value The probability of the presence or absence of disease given the results of a test – PVP is the probability of disease in a patient with a positive test result. – PVN is the probability of not having disease when the test result is negative.  Predictive Value DISEASE Present Absent True False Positive Positive Positive (TP) (FP) TEST False True Negative Negative Negative (FN) (TN) Predictive Value Positive (PVP) = TP/TP+FP Predictive Value Negative (PVN) =TN/TN+FN  Predictive value of a positive test Status of persons Affected Non-affected Positive A B A+B Test Negative C D C+D A+C B+D A+C+B+D PVP = A / (A+B) This is only valid for the sample of specimens tested  What factors influence the predictive value positive of a test.? Sensitivity? YES: To some extend. Specificity? YES: The more the test is specific, the more it will be negative for non affected persons. Thus, when the test is positive, it is probably truly positive (All non affected were correctly identified as testing negative). Prevalence of the disease? YES: Low prevalence: The test will pick up more false positives  Predictive value positive of a test according to prevalence and specificity 100 Specificity 90 80 PVP % 70 70% 60 80% 50 40 90% 30 95% 20 10 0 10 20 30 40 50 60 70 80 90 0 0 10 Pre vale nce (%) The predictive value of a negative test is the probability that an individual testing negative is truly non-affected Proportion of non-affected persons among those testing negative Predictive value of a negative test Persons testing negative (Negative by test) Persons non affected Persons affected (True negatives) (False negatives) Predictive value of a negative test = True negatives / Persons testing negative Estimate the 95% confidence interval  Predictive Value DISEASE Present Absent True False Positive Positive Positive (TP) (FP) TEST False True Negative Negative Negative (FN) (TN) Predictive Value Positive (PVP) = TP/TP+FP Predictive Value Negative (PVN) =TN/TN+FN  Predictive value of a negative test Status of persons Affected Non-affected Positive A B A+B Test Negative C D C+D A+C B+D A+C+B+D PVN = D / (C+D) This is only valid for the sample of specimens tested What factors influence the predictive value negative of a test? Sensitivity? YES: The more the test is sensitive, the more it captures affected persons. Thus, when the test is negative, it is probably truly negative (all affected were captured among the positive). Specificity? YES: But to a lesser extend. Prevalence of the disease? YES: Low prevalence: The test will pick up more true negatives Predictive value negative of a test according to prevalence and sensitivity 100 Sensitivity 90 80 70 60 70% PVN % 50 80% 40 90% 30 95% 20 10 0 10 20 30 40 50 60 70 80 90 0 0 10 Prevalence (%)  Relation between predictive values and (1) sensitivity and (2) specificity Se.P PVP  Se.P  (1  Sp)(1  P) Sp(1 - P)  PVN  Sp(1 - P)  (1 Se).P Positive (PPV) and negative (NPV) predictive values of a test according to the prevalence (95% sensitivity and specificity) 100 Predictive value (%) 80 PVN 60 40 20 PVP 0 0 25 50 75 100 Prevalence (%)  To whom predictive values matters Look at denominators! most? – Persons testing positive – Persons testing negative To clinicians and epidemiologists! Summary of QA Lecture 2 Classification types of diagnostic tests based on type of result obtained and method of analysis: qualitative, semi-quantitative and quantitative Accuracy and precision calculated used mean, standard deviation, %CV and % accuracy error/ bias. Diagnostic sensitivity, specificity, test efficiency, predictive values defined, calculated and results evaluated.  Summary Sensitivity and specificity matter to laboratory specialists – Studied on panels of positives and negatives – Look into the intrinsic characteristics of the test: Capacity to pick affected Capacity to pick non affected Predictive values matter to clinicians and epidemiologists – Studied on homogeneous populations – Look into the performance of the test in real life: What to make of a positive test What to make of a negative test