Streamlining Clinical Evaluation for Software as a Medical Device: Best Practices Under MDR 2017/745

The Medical Device Regulation (MDR) 2017/745, a regulation in the European Union (EU), governs medical devices, including Software as a Medical Device (SaMD)^[1]. This regulation classifies medical devices into four categories based on the significance of the information provided by a SaMD to healthcare decisions and the state of the patient’s healthcare situation or condition^[1].

The MDR requires software vendors to certify their products via notified bodies, which assess the conformity of the software before being placed on the market, with increasing levels of scrutiny based on the devices’ classification^[1]. This article aims to provide non-professionals with an understanding of theclinical evaluation process for SaMD under the MDR 2017/745, focusing on how to compile a clinical evaluation, including what clinical data are and what specifically needs to be considered for software as a medical device.

Understanding Clinical Evaluation for SaMD

The FDA has issued a guidance document titled “Software as a Medical Device (SaMD): Clinical Evaluation” to provide considerations for industry and FDA staff regarding the clinical evaluation of SaMD^[2]. Although this guidance does not impose regulatory requirements, it offers valuable insights into the process^[2].

Key Components of SaMD Clinical Evaluation

Clinical evaluation for SaMD involves assessing clinical safety, performance, and effectiveness based on clinical data^[3]. The process includes the following key components^[6]:

1. Valid Clinical Association (scientific validity)
2. Analytical / Technical Validation
3. Clinical Validation

The amount and quality of supporting data should be evaluated to determine and justify the level of clinical evidence, with sufficient quantity and quality being essential considerations^[10].

Clinical Evaluation Process

The clinical evaluation process for SaMD typically involves the following steps^[3]:

1. Identifying intended use and user needs
2. Reviewing available clinical data
3. Identifying gaps
4. Planning clinical investigations if necessary

Special considerations for AI/ML-based SaMDs include algorithm opacity and continuous learning^[6].

Compiling a Clinical Evaluation

When compiling a clinical evaluation for SaMD, it is important to consider the following:

• Clinical data: This includes any information related to the safety, performance, and effectiveness of the SaMD, such as scientific literature, clinical investigations, and real-world evidence^[9].
• Algorithm transparency: Manufacturers should provide evidence of the scientific validity, technical performance, and clinical performance of the algorithms used in the SaMD^[5].
• Intended use characteristics: The clinical evaluation should be tailored to the specific intended use and user needs of the SaMD^[6].
• Scope of evaluation: The clinical evaluation should cover all aspects of the SaMD, including its design, development, and post-market surveillance^[7].

Continuous Monitoring and Improvement

SaMD clinical evaluation should be an iterative and continuous process as part of the quality management system for medical devices^[7]. Real-world performance data can be used for multiple purposes, including monitoring clinical performance, improving effectiveness, and facilitating future releases^[10].

In summary, clinical evaluation for SaMD is a comprehensive process that involves assessing the safety, performance, and effectiveness of the software based on clinical data. By following the guidance provided by regulatory bodies and considering the unique aspects of SaMD, manufacturers can ensure that their products meet the necessary requirements and provide safe and effective healthcare solutions.

Key Requirements of MDR 2017/745 for SaMD

The Medical Device Regulation (MDR) 2017/745 introduces several key requirements for Software as a Medical Device (SaMD) to ensure safety, effectiveness, and continuous improvement throughout the product lifecycle. Here are the essential aspects manufacturers need to consider:

1. Clinical Evaluation Process

• • Manufacturers must establish a systematic and planned process to continuously generate, collect, analyze, and assess clinical data pertaining to their SaMD to verify its safety, performance, and clinical benefits when used as intended^[11].

1. • The clinical evaluation process should include determining the valid clinical association, technical performance, and clinical performance of the SaMD^[11].
   • Continuous monitoring of the SaMD’s safety, effectiveness, and performance is crucial, including active monitoring of real-world performance data for timely detection and correction of malfunctions, understanding user interactions, and improving effectiveness^[11].

2. Classification and Conformity Assessment

   • SaMD is classified under MDR rules 2, 10, 15, or 22, depending on its intended use and impact on patient health^[15].
   • Notified Bodies assess the conformity of medical devices, including SaMD, before they are placed on the market^[13].
   • The level of scrutiny applied by Notified Bodies increases based on the device’s classification^[1].

3. Technical Documentation and Standards

   • Manufacturers must maintain and update the technical file, risk management documentation, and other relevant information for their SaMD^[13].
   • The application of IEC 62304 is essential for SaMD, which includes establishing a Quality Management System (QMS), conducting Risk Management, and determining the Safety Software Classification^[15].
   • A gap analysis against IEC 62304 should be completed and maintained by SaMD manufacturers^[15].

4. Unique Device Identification (UDI) System

   • The UDI system applies to all medical devices, including SaMD, except custom-made and performance study/investigational devices^[16].
   • Manufacturers must assign a unique UDI to their SaMD, place the UDI carrier on the label or packaging, store the UDI, and maintain unique UDIs for their devices^[16].
   • The UDI consists of a UDI device identifier (UDI-DI) and a UDI production identifier (UDI-PI)^[16].

5. Post-Market Surveillance and Vigilance

   • MDR introduces increased post-market surveillance requirements for medical devices, including SaMD^[13].
   • Manufacturers must actively monitor and report any incidents, malfunctions, or adverse events related to their SaMD^[11].
   • Post-market clinical follow-up and continuous clinical evaluation are essential to ensure the ongoing safety and effectiveness of the SaMD^[13].

By adhering to these key requirements and following the guidance provided by regulatory bodies, SaMD manufacturers can ensure compliance with MDR 2017/745, maintain patient safety, and continuously improve their products based on real-world evidence.

Clinical Evaluation Process for SaMD

The clinical evaluation process for MDSW in the European Union involves determining valid clinical association, technical performance, and clinical performance^[11]. This process is crucial for ensuring the safety, effectiveness, and performance of the software throughout its lifecycle.

Key Steps in the Clinical Evaluation Process

The clinical evaluation process for SaMD includes several steps:

1. Defining the intended use and identifying the patient population^[12]
2. Developing a study protocol and conducting the study^[12]
3. Analyzing the results and submitting regulatory filings^[12]
4. Generating and assessing evidence based on the maturity of the underlying clinical association and confidence in the evidence as applied to the specific SaMD^[4]
5. Establishing a valid clinical association, which refers to the extent to which the SaMD’s output corresponds accurately to the real-world healthcare situation and condition identified in the SaMD definition statement^[4]
6. Conducting analytical validation to measure the ability of the SaMD to accurately, reliably, and precisely generate the intended technical output from the input data^[4]
7. Performingclinical validation to measure the ability of the SaMD to yield a clinically meaningful output associated with the target use in the identified healthcare situation or condition^[4]

Compiling a Clinical Evaluation for SaMD

When compiling a clinical evaluation for SaMD, it is essential to consider the following:

• Clinical data: This includes any information related to the safety, performance, and effectiveness of the SaMD, such as scientific literature, clinical investigations, and real-world evidence^[9].
• Specific considerations for software as a medical device: The clinical evaluation should take into account the unique aspects of SaMD, such as algorithmic complexity, evidence generation, technical and analytical validation, and clinical performance validation^[6].

Regulatory Requirements and Standards

Adhering to the requirements outlined in the MDR or IVDR, following the IEC 62304 standard, and complying with the ISO 14155 standard for clinical investigations enhances the likelihood of market success and contributes to improved patient outcomes^[12].

The Clinical Evaluation Report (CER) proves that the device performs as intended without compromising user safety and documents the clinical evaluation^[17]. Post-market Surveillance (PMS) is a systematic and proactive process for collecting information, assessing collected data, and managing events subject to the trend report^[17]. Post-market Clinical Follow-up (PMCF) specifies methods and procedures used to proactively collect and evaluate clinical data on a device’s performance and safety^[17].

Clinical investigations or clinical performance studies may be required to generate necessary data for compliance with General Safety and Performance Requirements (GSPRs)^[11]. The importance of an independent review of a SaMD’s clinical evaluation is highlighted, with the recommendation that the level of clinical evaluation and importance of independent review should be commensurate with the risk posed by the SaMD^[4].

Challenges and Considerations in Clinical Evaluation

Post-market surveillance and the Post-Market Surveillance Plan are crucial for monitoring the safety, effectiveness, and performance of the MDSW after it is placed on the market^[5]. However, healthcare facilities face increasing supply costs, with hospital executives frequently ranking these as primary concerns^[18]. Between 1991 and 2004, the cost of orthopedic implant devices rose by 132%, while hospital reimbursement rose by only 16%^[18]. Hospitals are becoming more aware that cost-effectiveness is critical in device selection, in addition to safety or physician preference^[18].

There is a growing demand for data on medical device efficacy and cost-effectiveness, but several obstacles prevent medical devices from undergoing the standard formulary committee review process^[18]. These challenges include:

1. Assessing product equivalencies for medical devices with different features developed by diverse manufacturers is problematic^[18].
2. Cost comparisons or effectiveness evaluations are not always possible because manufacturers rarely reveal pricing voluntarily^[18].
3. Surgeon preferences and vendor relationships significantly impact medical device selection^[18].

To address these challenges, alternative means of evaluating medical devices have been developed, such as:

• Value analysis teams (VATs): Many hospitals are developing VATs to evaluate new technology and justify the purchase and expense of these products^[18]. VATs facilitate decision making and standardization regarding medical devices and other supplies^[18].
• Payment caps and group purchasing organizations^[18].

Developers can pursue activities such as working with regulators early on, requesting to work with limited cases for rare conditions, and capturing user data as part of a quality management system^[9]. All SaMD need ongoing post-market surveillance, which can look at “real-world evidence” from the use of your device, including user data, complaints, and any adverse events^[9].

In summary, clinical evaluation for SaMD presents unique challenges, including cost-effectiveness, product equivalency assessment, and the impact of surgeon preferences and vendor relationships. Strategies such as value analysis teams, post-market surveillance, and capturing user data can help address these challenges and ensure the safety and effectiveness of SaMD throughout its lifecycle.

Conclusion

The clinical evaluation process for Software as a Medical Device (SaMD) is a comprehensive and ongoing process that ensures the safety, effectiveness, and performance of the software throughout its lifecycle. By following the guidance provided by regulatory bodies, such as theMDR 2017/745, and considering the unique aspects of SaMD, non-professionals can better understand the importance of compiling a thorough clinical evaluation.

This article has highlighted the key components of a clinical evaluation, including the assessment of clinical data, such as scientific literature, clinical investigations, and real-world evidence, as well as the specific considerations for SaMD, like algorithmic complexity and technical validation. By adhering to these principles and continuously monitoring the performance of SaMD through post-market surveillance, manufacturers can ensure that their products meet the necessary requirements and provide safe and effective healthcare solutions.