Quality Assurance & Quality Control PDF

QUALITY ASSURANCE & QUALITY CONTROL PART 1 BUT FIRST…FREQUENTLY ASKED QUESTIONS ABOUT THE QA/QC ASSIGNMENT PART 1 When is it due? Nov. 3rd Does it have to be forensics-related? No I want to write about [topic] but can’t find much online information. Should I still write my assignmen...

QUALITY ASSURANCE & QUALITY CONTROL PART 1 BUT FIRST…FREQUENTLY ASKED QUESTIONS ABOUT THE QA/QC ASSIGNMENT PART 1 When is it due? Nov. 3rd Does it have to be forensics-related? No I want to write about [topic] but can’t find much online information. Should I still write my assignment on it? There needs to be enough substance for you to relate your understanding of QA concepts. You can do some digging and see what you come up with; if not, have a backup plan. Also, ask yourself…Where online are you looking? Where else can you get information? OTHER UPDATES Grades Grades for the peer evaluation of resumes are visible You will have grades for Skills Inventory by the end of the week You will have grades for your edited resume/CV and letter before the drop date. I will make further announcements as new grades are released Market Research Assignment Remember to pair up with someone from your seminar to work on the assignment On the Discussion Board, under the appropriate thread, one member of each pairing must post their choice of organization/division/program for the Market Research Assignment along with the name of their partner QUALITY SYSTEMS Quality Control (QC) Most basic level of quality – making sure product or service is OK Does not ensure quality, only points out lack of quality Quality Assurance (QA) Aims to prevent non-conformities Considers related activities like training, document control and audits Ex. Horse in the stall QC is periodically checking to make sure the horse is in its stall QA is: Making sure that there is a functional lock on the stall door that all stable hands are trained to use Making sure that all staff understand the importance of locking the door when the horse is in the stall QUALITY SYSTEMS Quality Management Encompasses QA/QC Also includes quality improvements and aims to look at the quality management system as a whole Management system must contain Management (planning, goals, objectives) Resources (people, tools, equipment) Service/product realization (whatever is involved in creating the product) Monitoring (feedback loop)  Has a standard for quality management systems ISO 17025:2017 INTERNATIONAL Organizations can get ISO 17025:2017 ORGANIZATION FOR certified STANDARDIZATION Sector specific variations exist Published over 22,000 international standards covering almost every industry in existence including forensics QUALITY IN FORENSICS Forensic science by nature is a high stakes game Imperative that we get the right answer If for whatever reason we are not able to get the right answer it must be obvious that we have made an error Errors will happen – humans are imperfect therefore any system developed/run by humans will have capacity to make mistakes Type I Type II Our criminal justice system is built around the fact that it is better to have a false negative scenario than it is to imprison an innocent person Tied into the idea of innocent until proven guilty beyond reasonable doubt Quality management systems need to be designed such that Type I errors cannot happen  QUALITY MANAGEMENT FOR FORENSIC DNA TESTING LABS (E.G. FBI,CFS) Definitions, Goals & Objectives Analytical Procedures Scope Equipment Calibration & Maintenance Organization & Management Reports & Reviews Personnel Proficiency Testing Facilities Corrective Actions & Audits Evidence Control Safety Validation Outsourcing  Which labs are captured by the QA standards? For DNA, labs that utilize or upload data to CODIS (Combined DNA Index System) It does not preclude labs from doing stuff outside the scope of the QA program such SCOPE as research and development But that those activities need to be distinct i.e. You can’t use a brand new, never before tested DNA profiling process to generate profiles for CODIS ORGANIZATION AND MANAGEMENT Managerial staff must have authority and resources needed to carry out duties and meet requirements of QA Standards For example…specify and document responsibility, authority, and interrelation of personnel who manage/perform/verify work affecting validity of DNA analysis Technical leaders DNA analysts CODIS administrators  PERSONNEL Laboratory personnel shall have education, training and experience necessary Written job descriptions for all, that define responsibilities and skills Maintain records on relevant qualifications, training, skills and experience of technical personnel Documented training program Including a manual, practical exercises, participation in continuing education Probationary period and competency testing before completing independent casework Bottom line To work in a forensic lab there is A LOT of paperwork to prove you are qualified to be there And then you have to demonstrate it practically and consistently to keep your job FACILITIES Controlled access and suitable space For example…Space and flow set-up to not require movement of amplified DNA to pre-amplification work areas Decontamination procedures for all facilities and equipment EVIDENCE CONTROL Documented evidence control system to ensure integrity of all physical evidence Unique identifier, chain of custody (people, date, time), procedures to prevent contamination, controlled access areas Revisiting the example…. Work flow set-up to prevent movement of amplified DNA to pre-amplification work areas If possible retain/return a portion of evidence sample or extract Policy for disposition of evidence including policy on sample consumption  VALIDATION Confirmation, by examination and provision of objective evidence, that the particular requirements for a specified intended use are fulfilled The process where the scientific community acquires the necessary information to: Assess the ability of a procedure to obtain reliable results Determine the conditions under which results can be obtained Define the limitations of the procedure Demonstrating that a procedure follows the ‘3 Rs’ in the hands of the personnel performing the test Method validation The process of establishing the performance characteristics and limitations of a method and the identification of the influences which may change these characteristics and to what extent It is also the process of verifying that a method is fit for a purpose; i.e. for use for solving a particular analytical problem THE 3 RS Robust method – successful results are obtained a high percentage of the time and few, if any, samples need to be repeated Reliable method – the obtained results are accurate and correctly reflect the sample being tested Reproducible method – the same or very similar results are obtained each time a sample is tested  The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small variations in method parameters ROBUSTNESS Provides an indication of a method’s reliability during normal usage You do not want the method to fail when you only have enough material for a single try  Before introduction of a new method into routine use Whenever conditions change for WHEN IS which a method has been validated – e.g. a new instrument VALIDATION with different characteristics is NEEDED? bought Whenever the method is changed and the change is outside the original scope of the method  It is an important element of quality control WHY IS Validation helps provide assurance that a measurement will be METHOD reliable VALIDATION In some fields, validation of methods is a regulatory NECESSARY? requirement The validation of methods is good science  Developmental Validation – commonly performed by commercial manufacturers of novel methods or technologies (more extensive than internal validation) LEVELS OF Internal Validation – VALIDATION performed by individual labs when new methods are introduced Performance Checks – can be performed with every run (set of samples) DEVELOPMENTAL VALIDATION Demonstration of the accuracy, precision and reproducibility of a procedure Completed by the manufacturer, technical organization, academic institution, governmental laboratory, or other party creating something new Must precede the use of a novel methodology for forensic analysis PUBLICATION REQUIRED Peer-reviewed publication of the underlying scientific principle(s) of a technology is required Example: what are some of the underlying scientific principles of STR typing? DNA extraction PCR Fluorescent dye labels Capillary electrophoresis Run to run precision, enabling comparison to allelic ladders  Characterization Species of genetic specificity markers Sensitivity Stability studies studies EXAMPLES OF DNA-BASED Reproducibility Case-type DEVELOPMENTAL samples STUDIES Population Mixture studies studies Precision and PCR-based accuracy procedures INTERNAL VALIDATION The internal validation process should include the studies detailed below encompassing a total of at least 50 samples Known and nonprobative Mixture studies samples Contamination Reproducibility and precision Qualifying test Match criteria Sensitivity and stochastic studies PRECISION & ACCURACY SENSITIVITY Limit of Detection (LOD) – the lowest content that can be measured with reasonable statistical certainty Limit of Quantitative measurement (LOQ) – the lowest concentration of an analyte that can be determined with acceptable precision (repeatability) and accuracy under the stated conditions of the test  The ability of a method to measure only what it is intended to measure SPECIFICITY Ability to assess the analyte in the presence of components which may be expected to be present Typically these might include impurities, degradants, matrix, etc.  SYSTEM STABILITY Fitness of the sample or test to produce reliable results over time  Decide on Sensitivity, analytical resolution, requirements precision, etc. Plan a suite of experiments HOW DO YOU VALIDATE A Carry out the experiments METHOD? Use data to assess fitness for purpose Produce a Scope of the statement of method validation ASSUMPTIONS WHEN PERFORMING VALIDATION The equipment on which the work is being done is broadly suited to the application; it is clean, well maintained and calibrated The staff carrying out the validation are competent in the type of work involved There are no unusual fluctuations in laboratory conditions and there is no work being carried out in the immediate vicinity likely to cause interferences The samples being used in the validation study are known to be sufficiently stable TOOLS OF METHOD VALIDATION Standard samples Positive controls NIST standards Blanks Reference materials prepared in house and spikes Existing samples Statistics Common sense!  Concordance – are the same typing results obtained with the new technique as with an older DOES THE one? NEW METHOD Constant Monitoring – i.e. check multiple allelic ladders in IMPROVE a batch against one another to YOUR confirm precision CAPABILITY? Common Sense – are replicate tests repeatable?  Hundreds or thousands of samples are required to fully validate an instrument or method Each component of a test or process must be validated separately URBAN Validation should seek to understand LEGENDS OF everything that could potentially go wrong with an instrument or technique VALIDATION Validation is boring and should be performed by interns since it is beneath the dignity of a qualified analyst Once a validation study is complete you never have to revisit it VALIDATION (RETURNING TO DNA EXAMPLE) Labs may only use validated methodologies for DNA analyses Analysts must pass competency test on method before using it on case samples Performance of modified procedures must be compared with original procedure using similar DNA samples (quality, quantity, etc) Performance checks are required after any modifications or repairs to instruments or analysis software ANALYTICAL PROCEDURES All labs must have SOPs that are reviewed annually by technical leader in addition to any other laboratory audits All reagents (lot numbers, etc.) must be tracked used only for intended purpose Analytical procedures must be monitored through the use of controls Procedures must be checked annually or if substantial changes are made, against appropriate reference material Written guidelines for interpretation of data Verification of all controls Inclusion/exclusion policies Proper statistical interpretation of results Detection and control of contamination EQUIPMENT CALIBRATION AND MAINTENANCE Use equipment suitable for methods employed Documented program conducting performance checks and calibrations Minimum annual check Maintenance schedule Performance checks following repairs prior to being used in casework QUALITY CONTROL: Refers to the specific procedures that are done to make sure quality results are being produced Controls! Calibration A QC fail means that E.g. NIST references improvement is needed in the QA system AMERICAN NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY (NIST) Founded in 1901; now part of US Department of Commerce Authority on standards Allows for standardization across labs, industries, countries, etc. If you want to calibrate your scale using weights you get standard weights from NIST Have listings of available standards Barcode of Life Database (BOLD); based at Guelph University Smithsonian Institute Mammal Collection; mostly Suitland MD STANDARD OPERATION PROCEDURES (SOPs) Specific set of steps to accomplish a task (i.e. recipe) Integral part of a QA/QC system Provide consistency across multiple analysts, laboratories, etc. Exist to govern how any commonly repeated task is done in a lab environment Risk/liability reduction