Compass Health AI SOP: Usability Engineering (PDF)

Compass Health AI SOP: Usability Engineering Approvals Author Position Role Signature Date DD-MMM-YYYY Tenzin Head of Author 02-Jan-2024 Yangzom QA/RA James Approv...

Compass Health AI SOP: Usability Engineering Approvals Author Position Role Signature Date DD-MMM-YYYY Tenzin Head of Author 02-Jan-2024 Yangzom QA/RA James Approver 02-Jan-2024 02-Jan-2024 Baskin COO Rushabh Approver 02-Jan-2024 Head of Gudka Product Document# QMS-SOP-0011 Revision History Version Date Description 1.0 3-Jan- Initial Release 2024 Document# QMS-SOP-0011 1 Introduction 1.1 Purpose The primary purpose of this procedure is to provide instruction and a system to assign responsibilities for the evaluation of the usability characteristics of Compass Health’s (“Compass”) manufactured medical devices as they apply to issues of safety and for activating specific processes to ensure Compass Health’s manufactured devices are designed and developed with the intention of achieving reasonable device usability, the ultimate goal of which is to minimize use errors and use-associated risks. This procedure, which is developed to meet IEC 62366-1:2015, meets the medical device usability requirements defined in ISO 14971:2019. The protocol detailed herein is, therefore, to be understood as an essential feature of Compass Health’s core risk management activities and adhered to accordingly by relevant Compass Health personnel (see device’s Risk Management File of device in question and SOP: Risk Management, SOP: Product Development Process). 1.2 Scope This procedure applies to all activities involving the analysis, specification, design, verification and validation of the usability features of product developed by Compass Health. Document# QMS-SOP-0011 2 Applicable and Reference Documents 2.1 Applicable Standards The following standards and regulations are intended to be met by this procedure: IEC 62366-1:2015 Medical Devices – Application of usability engineering ISO 14971:2019 Medical Devices—Application of risk management to medical devices 2.2 Reference Documents Document Title Document # SOP: Competence & Training QMS-SOP-0005 SOP: Risk Management QMS-SOP-0013 SOP: Complaint Handling QMS-SOP-0026 SOP: Device Recall and Advisory Notices QMS-SOP-0028 SOP: Medical Device Reporting/ Mandatory Problem QMS-SOP-0029 Reporting SOP: Corrective and Preventative Action QMS-SOP-0031 3 Acronyms and Definitions Table 1: Acronyms Abbreviation Definitions Document# QMS-SOP-0011  MDD Directive for Medical Devices UEP Usability Engineering Process Table 2: Definitions Term Definitions Conscious, intentional act or intentional omission of an act that is counter to or violates NORMAL USE and is also beyond any further reasonable means of USER INTERFACE-related RISK CONTROL by Abnormal Use the MANUFACTURER EXAMPLES Reckless use or sabotage or intentional disregard of information for SAFETY are such acts. Materials accompanying a MEDICAL DEVICE and containing Accompanying information for the USER or those accountable for the installation, Documentation use and maintenance of the MEDICAL DEVICE, particularly regarding safe use. Correct Use NORMAL USE without USE ERROR Measure of accuracy and completeness with which USERS achieve Effectiveness specified goals. Efficiency EFFECTIVENESS in relation to the resources expended. Time period specified by the MANUFACTURER during which the Expected Service MEDICAL DEVICE is expected to remain safe for use (i.e. maintain Life basic SAFETY and essential performance) Document# QMS-SOP-0011  USER INTERFACE EVALUATION conducted with the intent to explore USER INTERFACE design strengths, weaknesses, and unanticipated USE ERRORS Formative Evaluation Note 1: FORMATIVE EVALUATION is generally performed iteratively throughout the design and development PROCESS, but prior to SUMMATIVE EVALUATION, to guide USER INTERFACE design as necessary. Hazard Potential source of harm. USE SCENARIO that could lead to a HAZARDOUS SITUATION or HARM Note 1: A HAZARD-RELATED USE SCENARIO can often be linked to a Hazard Related potential USE ERROR. Use Scenario Note 2: A HAZARD-RELATED USE SCENARIO is not related to a failure of the MEDICAL DEVICE, unless the MEDICAL DEVICE failure was caused by a USE ERROR. Intended purpose; use for which a product, process or service is Intended Use intended according to the specifications, instructions and information provided by the manufacturer. The natural or legal person with responsibility for the design, manufacture, packaging and labeling of a device before it is placed Manufacturer on the market under its own name, regardless of whether these operations are carried out by this entity or on its behalf by a third party. Any instrument, apparatus, implement, machine, appliance, Medical Device implant, in vitro reagent or calibrator, software, material or other similar or related article, intended by the MANUFACTURER to be Document# QMS-SOP-0011  used, alone or in combination, for human beings for one or more of the specific purpose(s) of diagnosis, prevention, monitoring, treatment or alleviation of disease, diagnosis, monitoring, treatment, alleviation of or compensation for an injury, investigation, replacement, modification, or support of the anatomy or of a physiological process, supporting or sustaining life, control of conception, disinfection of MEDICAL DEVICES, providing information for medical purposes by means of in vitro examination of specimens derived from the human body, and which does not achieve its primary intended action in or on the human body by pharmacological, immunological or metabolic means, but which can be assisted in its function by such means. Operation, including routine inspection and adjustments by any USER, and stand-by, according to the instructions for use or in accordance with generally accepted practice for those MEDICAL DEVICES provided without instructions for use. NOTE: USE ERROR can occur in NORMAL USE. Normal use NOTE: NORMAL USE should not be confused with INTENDED USE. While both include the concept of use as intended by the MANUFACTURER, INTENDED USE focuses on the medical purpose while NORMAL USE incorporates not only the medical purpose, but maintenance, transport, etc. as well. Objective Data supporting the existence or verity of something, which may be Evidence derived from observation, measurement, testing, and so forth. Document# QMS-SOP-0011  Living being (person) undergoing a medical, surgical or dental Patient PROCEDURE Function that involves USER interaction that is related to the SAFETY of the MEDICAL DEVICE. Note 1: Often a PRIMARY OPERATING FUNCTION is interacted with by a series of TASKS that can be broken down into a series of USER Primary interactions. Operating Function Note 2: The concept of SAFETY includes loss or degradation of performance resulting in an unacceptable RISK to the PATIENT, including USE ERROR that prevents the USER from effectively using the MEDICAL DEVICE to achieve its intended medical purpose. In IEC 60601-1, this is referred to as ‘essential performance’. Entity accountable for the use and maintenance of a MEDICAL Responsible DEVICE or combination of MEDICAL DEVICE such as a hospital, an Organization individual clinician or a lay person. Combination of the probability of occurrence of harm and the Risk severity of that harm. Systematic use of available information to identify hazards and to Risk Analysis estimate the risk. Process in which decisions are made and measures implemented by Risk Control which risks are reduced to, or maintained within, specified levels. USER INTERFACE EVALUATION conducted at the end of the USER Summative INTERFACE development with the intent to obtain OBJECTIVE Evaluation EVIDENCE that the USER INTERFACE can be used safely Document# QMS-SOP-0011  Note: SUMMATIVE EVALUATION relates to validating the safe use of the USER INTERFACE One or more USER interactions with a MEDICAL DEVICE to achieve a desired result Note 1 to entry: A TASK description should include the allocation of activities and operational steps between the Task USER and the MEDICAL DEVICE. Note 2 to entry: TASKS should not be described solely in terms of the functions or features provided by the MEDICAL DEVICE. User Interface of USER INTERFACE or part of a USER INTERFACE of a MEDICAL Unknown DEVICE previously developed for which adequate RECORDS of the Provenance USABILITY ENGINEERING PROCESS of this standard are not (UOUP) available Characteristic of the USER INTERFACE that establishes Usability EFFECTIVENESS, EFFICIENCY, ease of USER learning and USER satisfaction. Application of knowledge about human behaviour, abilities, Usability limitations, and other characteristics related to the design of tools, Engineering devices, systems, tasks, jobs, and environments to achieve adequate USABILITY. Usability Set of RECORDS and other documents that are produced by the Engineering File USABILITY ENGINEERING PROCESS. Testable requirements for USABILITY of the PRIMARY OPERATING Usability Goals FUNCTIONS Document# QMS-SOP-0011  Method for exploring or evaluating a USER INTERFACE with intended Usability Test USERS within a specified intended USE ENVIRONMENT Actual conditions and setting in which USERS interact with the MEDICAL DEVICE Note: The conditions of use or attributes of the USE ENVIRONMENT Use Environment can include hygienic requirements, frequency of use, location, lighting, noise, temperature, mobility, and degree of internationalization. Specific sequence of TASKS performed by a specific USER in a Use Scenario specific USE ENVIRONMENT and any resulting response of the MEDICAL DEVICE Summary of the important characteristics related to the context of use of the MEDICAL DEVICE Note 1: The intended medical indication, PATIENT population, part of the body or type of tissue interacted with, USER PROFILE, USE Use Specification ENVIRONMENT, and operating principle are typical elements of the (APPLICATION USE SPECIFICATION. SPECIFICATION) Note 2: The summary of the MEDICAL DEVICE USE SPECIFICATION is referred to by some authorities having jurisdiction as the ‘statement of intended use’. Note 3: The USE SPECIFICATION is an input to determining the INTENDED USE of ISO 14971:2019 User Interface Collection of specifications that comprehensively and prospectively Specification describe the USER INTERFACE of a MEDICAL DEVICE USER action or lack of USER action while using the MEDICAL DEVICE that leads to a different result than that intended by the Use Error MANUFACTURER or expected by the USER NOTE: USE ERROR includes slips, lapses, and mistakes. Document# QMS-SOP-0011  Person using, i.e. operating or handling, the MEDICAL DEVICE. NOTE 1: This includes, but is not limited to, cleaners, maintainers User and installers. NOTE 2: PATIENTS or other laypersons can be USERS. Subset of intended USERS who are differentiated from other intended USERS by factors that are likely to influence USABILITY, User Group such as age, culture, expertise or type of interaction with a MEDICAL DEVICE User Interface Means by which the USER and the MEDICAL DEVICE interact. PROCESS by which the MANUFACTURER explores or assesses the USER interactions with the USER INTERFACE. Note 1 to entry: A USER INTERFACE EVALUATION may consist of one or more of the following techniques, amongst others, USABILITY TESTS, expert reviews, heuristic analyses, design audits or a cognitive walk through. User Interface Evaluation Note 2 to entry: USER INTERFACE EVALUATION is frequently performed iteratively throughout the design and development PROCESS. Note 3 to entry: USER INTERFACE EVALUATION is a part of the activities involved in verifying and validating the overall MEDICAL DEVICE design. Summary of the mental, physical and demographic traits of an intended USER population, as well as any special characteristics User Profile that can have a bearing on design decisions, such as occupational skills and job requirements. Document# QMS-SOP-0011  Confirmation, through the provision of OBJECTIVE EVIDENCE, that Validation the requirements for a specific INTENDED USE or application have been fulfilled 4 Process Roles Title/Functional Responsibilities Role Planning verification, validation, monitoring, inspection and test activities specific to the product and the criteria for Quality Assurance product acceptance specific to usability engineering-related activities and the associated risk management activities. Managing all usability engineering-related issues and ensuring adequate documentation coverage Completing (customized per product) 935-01016 Template: Validation Plan (see 6.8.2) and 935-01017 Template: Validation Report Assigning usability engineering activities to project team Product Manager members as needed. Presenting a summary of the Usability Engineering File / product to Executive Management as needed. Advise and escalate issues as required. Tracing (via traceability matrix) UEP design transfer to production and for planning and specifying production processes. Project Team Members Following the Usability Engineering process in this SOP Assigned Usability Following the Validation Plan for each product Engineering Performing and documenting activities as assigned by the Activities Project Manager Document# QMS-SOP-0011  Analyzing system modules to the degree of detail required to identify the appropriate mitigations 5 Usability Engineering Process 5.1 General requirements 5.1.1 Maintenance of the Usability Engineering Process (UEP) 5.1.1.1 In accordance with IEC 62366, Compass Health has established and documented (and shall implement and maintain) a usability engineering process in order to provide safety for the patient, user and others using Compass Health’s medical devices. 5.1.1.2 The contents of this UEP operational procedure shall address issues pertaining to user interactions with the device according to the information contained in the accompanying document(s) which shall include, (but is not necessarily limited to): transport; storage; installation; operation; maintenance and repair; disposal 5.1.1.3 Compass Health shall implement the usability engineering process for all of its devices, which includes validation activities (with a dedicated focus on usability, both formative and summative evaluation), throughout the product lifecycle to end-of-life. All usability activities shall be planned, carried out and documented by a competent personnel. Note: Subclause 6.2 of ISO 13485:2016 contains additional information relating to personnel competence. Note: Activities described in Clause 5 of IEC 62366:2015 are described in logical order in Figure A.4, but may be carried out in a flexible order as appropriate. 5.1.1.4 Compass Health QA shall maintain and store all documentation relating to the Usability Engineering Process as evidence in the Usability Engineering File (UEF) for each product. Evidence shall be made readily available upon request by relevant Notified Document# QMS-SOP-0011 Bodies, Competent National Authorities, and the like for conformity assessment purposes or any other regulatory activities. 5.1.2 Residual Risk/Risk Control 5.1.2.1 Compass Health shall review residual risks, including risks associated with the user interface design, of all its devices. 5.1.2.2 To reduce use-related risks associated with Compass Health products, Compass shall use one or more of the following options: 1. Ensure inherent safety by Design; 2. Implement protective measures in the device itself or in the manufacturing process 3. Provide information for Safety 5.1.2.3 Compass Health shall ensure that the residual risk(s) associated with the usability of a medical device has been adequately reviewed and either determined: To be within an acceptable range by maintaining compliance with the usability engineering process detailed herein, meeting the acceptance criteria documented in the Validation Plan (specifically the user interface evaluation plan section of the Validation Plan, see section 6.9.2) and evaluating objective evidence on an ongoing basis as it becomes available. To not be within an acceptable range and therefore managed as per the planned arrangements documented in the SOP: Risk Management (QMS-SOP-0022). 5.1.2.4. If objective evidence emerges at any point subsequent to the initial assessment of the residual risk associated with the usability of the device in question that indicates the level of residual risk has risen above an acceptable range (see corresponding Risk Management File), Compass Health shall, if judged necessary to reduce the level of risk, institute one of the options listed above in section 5.1.2.2 as required. 5.1.2.5. Likewise, if design changes are indeed implemented as a result of the emergence of increased risks relating to the usability of the device in question –or for whatever reason, for that matter Compass Health shall, as per ISO 14971:2019, Section 7.6, determine whether additional hazardous situations have been generated as a by-product of the implementation of said corrective action and take the necessary steps to eliminate them if they are deemed to represent an unacceptable increase in the level of residual risk associated with the device (see SOP: Risk Management and corresponding Risk Management File). Document# QMS-SOP-0011 5.1.2.6 Compass Health shall document activities relating to the assessment of residual risks, including risks associated with the usability of the device in question, in the device’s Risk Management Plan and Report (located in the corresponding DHF/Risk Management File). 5.1.2.7 Compass Health shall perform reviews periodically of various data sources including post-market surveillance activities (see SOP: Post-Market Surveillance and include relevant findings in design and usability considerations for existing and future products. *The process for evaluating residual risks associated with a device manufactured by Compass Health is outlined in SOP: Risk Management and the device’s individual Risk Management File. 5.1.3 Information for Safety as it relates to Usability 5.1.3.1 Compass Health shall use the Information for Safety of each Compass Health- made medical device as a risk control measure as stipulated in the device’s individual Risk Management File. 5.1.3.2 Compass Health shall subject said information –which may include warnings and limitations in the instructions for use/ accompanying document(s), the display of a monitored variable, training materials, maintenance details, or on any relevant markings (i.e. including the use of international symbols specific to warnings) —to the usability engineering process. 5.1.3.3 Compass Health shall document activities relating to the incorporation of said information into the usability engineering process in the device’s individual Usability Engineering File. *Information for Safety is one of three possible elements in a manufacturer’s hierarchical approach to Risk Control (see ISO 14971: 2019, 7.1). The remaining two control measures are a) inherent safety by design, and b) protective measures in the medical device itself or in the manufacturing process, e.g. Alarm systems. Refer to the Risk Management File of the product in question for information on the device’s risk control strategies. 5.2 Usability Engineering File (UEF) 5.2.1 Compass Health shall record for each Compass Health-manufactured medical device the results of the usability engineering process in addition to the relevant elements Document# QMS-SOP-0011 of the process stated throughout this document. Compass Health shall record this information in the device’s Usability Engineering File or, if the information is stored elsewhere, provide reference therein to its place of storage, e.g., create an index as part of the Usability Engineering Plan and Report which identifies the location of the information and / or supporting documents relevant to the Usability Engineering Process. 5.3 Tailoring of the Usability Engineering Effort 5.3.1 The level of effort and the choice of methods and tools used to perform the usability engineering process for medical products at Compass Health shall be commensurate with the nature and risk level of the device in question and may be based on: 4. Size and complexity of the user interface 5. Severity of the harm associated with the use of the medical device 6. Extent or complexity of the use specification 7. The presence of user interface of unknown provenance 8. the extent of the modification to an existing medical device user interface that had been subjected to the usability engineering process Any Tailoring decisions will be documented in the Usability Engineering Plan and Report. 5.3.2 If deemed appropriate to modify the design of the device for any reason including to enhance its usability, the effort of the usability engineering process may be scaled up or down based on the significance of the proposed modification as determined by the results of a risk analysis (see device’s Risk Management File and SOP: Risk Management (QMS- SOP-0022)). 6 Usability Engineering Process 6.1 Prepare Use Specification 6.1.1 Compass Health shall provide details on the given device’s application in the device’s usability engineering file (specifically in the Usability Engineering Plan and Report) according to the following specifications: Its intended medical indication, whether it be to screen, monitor, treat, diagnose, or prevent one or more specific disease(s)/ condition(s), etc.; Its intended target patient population categorized by age, weight, level of health, condition, etc.; Document# QMS-SOP-0011  Its intended part of the body or type of tissue applied to or interacted with; Its intended user profile (see Definitions above); Its use environment such as those specific to the environment wherein the device shall be used (including hygienic requirements), user interface, frequency of use, location, mobility, and so forth; and A description of its operating principle. 6.2 Identification of Primary Operating Functions 6.2.1 Compass Health shall identify via a functional analysis (see sub-clause 5.2 of IEC 62366:2015) or a preferred alternative method the primary operating functions of the device in question used frequently by the user and record them in the Usability Engineering Plan and Report. Note – this activity is not required as part of compliance to IEC 62366-1:2015. 6.2.2 The primary operating functions inputs shall include the following: frequently used user functions; user interface of the frequently user functions, and functions related to the safety characteristics of the device. 6.3 Identification of User Interface characteristics related to SAFETY and potential USE ERRORS 6.3.1 Compass Health shall identify the user interface characteristics of the device in question that have a bearing on safety relating to the product’s usability as part of the risk analysis process and in keeping with ISO 14971:2019 requirements (refer to SOP: Risk Management and the device’s Risk Management File) 6.3.2 During the identification process, the following elements shall be considered: Use Specification, including user profile(s) (see definitions above) User interface; and Frequently used functions. 6.3.3 Compass Health shall record the results of this process in the device’s Usability Engineering File/ Usability Engineering Plan and Report. Document# QMS-SOP-0011 6.4 Identification of Known or Foreseeable Hazards and Hazardous Situations 6.4.1 Compass Health shall also identify known or foreseeable hazards associated with the usability of all Compass Health-manufactured devices as part of the risk analysis process in keeping with ISO 14971:2019(sub clause 5.4) requirements (refer to SOP: Risk Management and the individual device’s Risk Management File). 6.4.2 Compass Health shall consider hazards to patients, users and other persons as part of the hazard identification process. 6.4.3 Reasonably foreseeable sequences or combinations of events involving the user interface that may result in a hazardous situation associated with the medical device shall be identified according to the protocol established in SOP: Risk Management, the individual device’s Risk Management File and this document –and the severity of the resulting possible harm shall be determined. 6.4.4 During the hazard identification process relating to product usability, the following elements shall be considered: Use Specification, including user profile(s) (see definitions above); task-related requirements; context of use; information on hazards and hazardous situations known for existing user interfaces of medical devices of a similar type from post-market surveillance activities (SOP: Post-Market Surveillance); Identified use errors; preliminary use scenarios; possible use errors (see C.2 in IEC 62366:2015 for examples); if an incorrect mental model of the operation of the medical device can cause a use error resulting in a hazardous situation; and results of the review of the user interface. 6.4.5 Compass Health QA and/or designee shall record the findings of the hazard and hazardous situations identification activities relating to device usability, including the results of the corresponding severity assessment(s), as defined in the SOP: Risk Management. Document# QMS-SOP-0011 6.4.6 The results of the evaluation shall also be included in the appropriate engineering documentation (requirements and design specifications) in the specified Usability sub- section of each document/ device in question. 6.4.7 Compass Health shall also note any abnormal use conditions identified throughout the procedure and feed this information into the risk management process (refer to SOP: Risk Management and the device’s Risk Management File). 6.5 Identify and Describe Hazard-Related Use Scenarios 6.5.1 Compass Health shall identify and describe reasonably foreseeable hazard-related use scenarios associated with the identified hazards and hazardous situations as documented in the device’s Risk Management File. 6.5.2 The description of each hazard-related use scenario shall include all tasks, sequence of tasks and an assessment of the severity of the associated harm. 6.5.3 Compass Health shall select the hazard-related use scenarios to be included in the summative evaluation. Compass Health shall select: All hazard-related use scenarios; or the subset of the hazard-related use scenarios based on the severity of the potential harm that could be caused by use error 6.5.4 The rationale of the choice of the scheme used to select the hazard-related use scenarios and the results of applying it shall be stored in the usability engineering file. 6.6 Establish User Interface Specification 6.6.1 Compass Health shall develop a User Interface Specification for each of its devices. The User Interface Specification shall be documented in the appropriate engineering documentation (System/Software Requirements Specification, risk management file or a unique User Design Specification). 6.6.2 The User interface specification will detail: a) the use specification; b) the known or foreseeable use errors associated with the Medical Device; and c) the hazard-related use scenarios Document# QMS-SOP-0011 6.6.3 The user interface specification shall include: a) testable technical requirements relevant to the user interface, including the requirements for those parts of the USER INTERFACE associated with the selected RISK CONTROL measures; b) an indication as to whether ACCOMPANYING DOCUMENTATION is required; and c) an indication as to whether MEDICAL DEVICE-specific training is required. 6.6.3 Compass Health shall treat the following elements as User Interface Specification inputs: application/use specification (as per 62366:2015); primary operating functions (as per 62366:2015); context of use Specification The hazard-related use scenarios and known or foreseeable use errors associated with the medical device. 6.6.4 The usability specification shall describe, at minimum: use scenarios relating to the primary operating functions, including; o frequent use scenarios o reasonably foreseeable worst-case use scenarios; user interface requirements for the primary operating functions, including those designed to mitigate risk (see corresponding Risk Management File); requirements for determining whether the primary operating functions are easily recognizable by the user. Testable technical requirements relevant to the user interface including those associated with risk control measures. An indication if user documentation and specific training is required 6.6.5 Compass Health QA shall record the User Interface Specification relating to the device in question in the device’s engineering documentation as appropriate and a summary shall be documented in the Usability Engineering File/Usability Engineering Plan and Report. Document# QMS-SOP-0011 6.7 Usability Validation 6.7.1 General 6.7.1.1 Compass Health shall evaluate and validate the usability of the device in question—which shall entail careful scrutiny of the accompanying document, a key component of the user interface—according to the dedicated usability section of the Validation Plan (see section 6.8.2). 6.7.1.2 Compass Health shall assign this task to individuals who were not directly responsible for the user interface design. This can include beta user interface design validation with a trial customer. 6.7.1.3 Compass Health QA shall record the results of the validation activities in the Validation Report / device. In keeping with ISO 14971:2019 7.3, Compass Health shall use the results of the Validation Process in the evaluation of the overall residual risks associated with the device in question (refer to SOP: Risk Management and the device’s Risk Management File). 6.7.1.4 If the acceptance criteria documented in the Validation Plan is not met: further user interface design and implementation activities shall be performed (see 5.6); or (if further improvement is deemed impracticable) Compass Health shall gather and review relevant data and literature according to the procedure outlined in SOP: Post-Market Surveillance to determine via formal risk estimation activities (refer to SOP: Risk Management and the device’s Risk Management File) if the medical benefits of the intended use outweigh the risk arising from usability problems. * *If the evidence accumulated as a result of this action does not support the conclusion that the medical benefits outweigh the risk, then the level of risk shall be deemed unacceptable. Note: refer also to sections D.5.13 and D.5.15 in IEC 62366:2015 for additional guidance on performing a usability validation. 6.7.2 Establish User Interface Evaluation (Usability Validation) Plan 6.7.2.1 Compass Health shall prepare and maintain a user interface evaluation plan for the user interface specification. The Validation Plan shall specify: Document# QMS-SOP-0011  the objective of any formative evaluations and summative evaluations; the method(s) used to validate the usability of the primary operating functions; the criteria* for determining successful validation of the usability of the primary operating functions based on the usability specification; and the involvement of representative intended users. 6.7.2.2 If usability tests are employed, Compass Health shall document the involvement of the representative intended users and user profile to which they belong. Multiple user profiles may be combined into a user group for the purposes of a usability test. document the test environment and other conditions of use. based on the use specification specify whether accompanying documentation is provided during the test specify whether medical device-specific training is provided prior to the test and the minimum elapsed time between the training and the beginning of the test 6.7.2.3 Compass Health shall employ qualitative or quantitative methods (or any combination thereof depending on the nature of the design of the device and its intended use) when validating a given device’s usability. 6.7.2.4 Compass Health shall perform the validation activities in either a simulated use environment (internal or external to the office), or in the environment wherein the device is intended to be used in practice. This can include validation with a trial customer. 6.7.2.5 Compass Health QA-RA shall maintain the User Interface Evaluation Plan in the DHF and/or the Risk Management File (as defined in the Plan itself) and reference it as part of the usability engineering file. 6.7.3 Formative Evaluation Planning Formative usability evaluation shall be incorporated throughout the device's development, verification and validation. Formative Usability evaluation shall also be conducted iteratively to refine and ultimately optimize the user interface based on inputs from the formative evaluation results. Compass shall apply focus and effort to formative evaluations early on, as the information derived from such evaluations are valuable inputs to the design and development process. The user interface evaluation plan for formative evaluation shall address: Document# QMS-SOP-0011  Objective of the evaluation Participant selection in accordance with the identified, distinct user groups Part of the USER INTERFACE being evaluated Training (withheld, delivered, or both) Learning decay period between (1) training and (2) device use in a usability test session Simulated environment equivalency to the actual environment Data collection approach and tools Post-test analyses to be performed on such data as use errors, close calls, difficulties, anecdotal comments, including responses to interview questions. 6.7.4 Summative Evaluation Planning Summative Usability Evaluation shall commence upon completion of iterative formative usability evaluation(s). Planning for Summative Evaluation will likely not be finalized until after formative evaluation has been completed. Compass shall perform a summative evaluation of the overall usability of the product through pilot / clinical studies. Subsequently, for any new feature and/or User Interface(UI) that Compass wishes to introduce into their product for clinical use, Compass shall determine whether the design of the feature in evaluation would benefit from formative and summative usability testing to maximize user adoption and usability. The Product Manager assigned shall aim to follow the usability engineering process for all new features and/or User interface (UI). However, it is permissible for minor UI changes to not undergo summative usability evaluation, as usability of the new UI would be evaluated through the device's verification and validation testing. Summative usability evaluation plan shall include with representative users and tasks designed to measure the usability (defined as effectiveness, efficiency and satisfaction) of the complete product. A Summative Evaluation can include training as part of the protocol, as appropriate, to simulate realistic use. An appropriate wait time might be needed between the training and the rest of the Summative Evaluation to allow for representative learning decay. Compass Health may perform the summative evaluation as a single evaluation or in multiple evaluations. Guidance on the evaluation of the adequacy of risk control measures can be found in ISO 14971:2019, Clause B. Document# QMS-SOP-0011 The user interface evaluation plan for summative evaluation shall specify, similar to formative usability evaluation: Objective of the evaluation Participant selection in accordance with the identified, distinct user groups Part of the USER INTERFACE being evaluated Training (withheld, delivered, or both) Learning decay period between (1) training and (2) device use in a usability test session Simulated environment equivalency to the actual environment Data collection approach and tools Post-test analyses to be performed on such data as use errors, close calls, difficulties, anecdotal comments, including responses to interview questions. 6.8 User Interface Design, Implementation and Formative Evaluation 6.8.1 Compass Health shall design and implement the user interface as described in the device’s usability section of the engineering specification utilizing, as appropriate, usability engineering methods and techniques (including formative evaluation) and including accompanying documentation and training capability, if needed. 6.8.2 The results of the formative evaluation shall be stored in the usability engineering file. 6.8.3 Compass Health shall modify the design of the user interface at any point during its lifecycle in the event information comes to light via post-market surveillance activities (or any other means based on objective evidence indicating an increase of risk associated with the user interface design such where new use errors, hazards, hazardous situations or hazard-related use scenarios are discovered). Compass Health shall repeat the steps of Clause 5 in IEC 62366-1:2015 as appropriate. 6.8.4 If the level of risk is indeed judged to be unreasonable, relevant recall, reporting and corrective actions may be implemented by Compass Health, as apply, based on the nature of the circumstances leading to the modification and the point at which the discovery is made in the product’s lifecycle (see SOP: Corrective and Preventive Action, SOP: Device Recall and Advisory Notices, and for the USA and Canada – SOP: Medical Device Reporting/ Mandatory Device Reporting). Document# QMS-SOP-0011 6.9 Perform Summative Evaluation of the Usability of the User Interface 6.9.1 Compass Health shall perform summative evaluation of each hazard-related use scenario on the final or production equivalent user interface according to the user interface evaluation plan. The results shall be stored in the usability engineering file. 6.9.2 If new use errors, hazards, hazardous situations or hazard-related use scenarios are discovered during the analysis of data from the summative evaluation: If yes, Compass Health shall repeat the activities of Clause 5 in IEC 62366-1:2015 as appropriate; If not, Compass Health shall determine whether further improvement of the user interface design as it relates to safety is necessary and practicable o If yes, Compass Health shall re-enter the usability engineering process at Clause 5.6 in IEC 62366-1:2015; o If not, Compass Health shall: ▪ document why improvement is not practicable; ▪ identify the data from the usability engineering process needed to determine the residual risk related to use; and ▪ evaluate the residual risk according to Clause 6.4 in ISO 14871:2007 6.9.3 Usability of a medical device as it relates to safety is presumed to be acceptable if the usability engineering process detailed in IEC 62366-1:2015 has been complied with. 6.10 User Interface of Unknown Provenance Instead of all the requirements of 5.1 through 5.9, UOUP may be evaluated according to Annex C. 7 Accompanying Document 7.1 Compass Health shall include a summary of the Use Specification (see 6.1) of the device in question (the statement of intended use) in the accompanying document (see definitions above). 7.2 Compass Health shall include a concise description of the device in the accompanying document to allow the user to develop an accurate mental model of the device. This description shall detail, where relevant: Document# QMS-SOP-0011  the device’s operating principle; the device’s significant physical characteristics; the device’s significant performance characteristics; and the device’s intended user profile. 7.3 Compass Health shall include information pertaining to training and training materials in the accompanying document, if required. 7.4 The accompanying document shall be written at a level commensurate with the level of education and training of intended users. Documents shall be translated in keeping with the governing regulations per country (as defined in SOP: Labeling and Packaging [QMS- SOP-0027]). 7.5 Compass Health shall provide the accompanying document electronically, if appropriate. When it decides to do so, Compass Health shall determine which information provided electronically should also be provided in hard copy form or communicated through markings on the device itself, e.g., information relating to important safety and performance features of the device, use of international symbols as defined in EN 980. 8 Training and Materials for Training 8.1 If training on the device in question is required*, Compass Health shall perform one of the following actions: provide the materials necessary for training (translations as appropriate); ensure that the materials necessary for training are available; make the training available; or make training available to the responsible organization that enables it to train its users 8.2 When training is deemed required, Compass Health shall describe the available training options along with the suggested duration and frequency of such training in the accompanying document (see section 7). 8.3 Compass Health shall base the training and training material(s) on the device’s intended use and its corresponding user profile(s). *Training is deemed required when there is a reasonable likelihood that the safe and effective use of the device’s primary operating functions by the intended user may be Document# QMS-SOP-0011 compromised in the absence of training. It is not expected that training will be required for Compass Health’s products. 9 Quality Records Record type Description Usability Identification of strategies and approaches for validation Evaluation/Validation Plan of requirements Usability Records of validation conducted Evaluation/Validation Report Usability Engineering Plan See section 6.1.1 and Report Document# QMS-SOP-0011