FDA’s Regulation of AI/ML SaMD

Software is playing an increasingly important role in the medical device space. Whether as a cloud-based analytic platform, a digital health app on your phone, or a clinical decision support assistant for clinicians, software-as-a-medical-device (SaMD) has caught fire. Why? Because:

•  Healthcare is becoming increasing digitized, so there is a rapidly growing mountain of clinical data.
• Software development is quick, in most cases much quicker than the traditional development of “hardware” medical devices.
• We carry with us processing power in our phones and our laptops that can run complex software in real time–software that can produce meaningful, actionable outputs that professionals can use immediately to take better care of their patients.

But the rules of behavior are not yet set for this brave new SaMD world. For example, when is a software program an “app” and when is it a medical device? If it’s a medical device, how do we know that it’s safe and effective at doing what it claims to do? And who is going to oversee and guide this burgeoning field, controlling risks without stifling innovation?

The US Food and Drug Administration (FDA) is charged with the oversight of all medical devices introduced into interstate commerce in the USA. The FDA works under the strict definitions found in its guiding statute, the Federal Food, Drug, and Cosmetic Act (the Act), which in 1976 defined what is and is not a medical device.1

Of course, any software that meets the Act’s definition of a medical device will indeed come under FDA’s purview. But there are specific qualities to a SaMD that make it unique, that break the mold when it comes to medical device regulation and to determining a reasonable assurance of safety and effectiveness–the FDA’s benchmark for evaluating devices. For example, SaMD is a medical device that must stand alone and not be part of a hardware medical device.2 SaMD is also something that can be “hacked”. It can malfunction in new and different ways that are difficult, if not impossible, to understand and that could even modify itself on the fly as it gains experience through use. These unique properties have created new and significant challenges to regulatory scientists, and their field is evolving rapidly.

One SaMD subset that is becoming more and more common is the “artificial intelligence/machine learning (AI/ML)-powered SaMD”. These medical devices are powered by complex algorithms that intake medical data such as physiological signals, biomarkers, and images, and produce outputs and insights that humans cannot–or can only do so slowly and imprecisely. AI/ML SaMDs are developed (that is, “trained”, “tuned”, and “validated”) using very large quantities of data–sometimes millions or billions of records. They excel at tasks such as pattern recognition, providing clinicians with powerful decision support in real time as they care for their patients. For example, there are now several AI/ML SaMDs that can identify an area of an ultrasound or x-ray image that is suspicious for breast, thyroid, or lung cancer. There are others that can determine from an electrocardiogram that a patient’s heart does not pump normally. Some can even look at a patient’s electronic medical record and offer diagnoses or suggest tests that are much better tailored to the specific patient.

 

Regulatory Challenges

The FDA and the International Medical Device Regulators Forum has been giving AI/ML SaMD a lot of thought over the past 5 years. They have developed frameworks through which we can understand and manage the risks and benefits of AI/ML SaMDs and develop the regulatory science for regulating them.3 Some unique SaMD regulatory challenges are:

• Data Quality and Quantity: SaMDs require large amounts of high-quality, labeled data for training, testing, and validation. Limited or biased data can lead to inaccurate or unsafe outcomes.
• Bias and Fairness: Training data that is biased can perpetuate or exacerbate already-existing biases in healthcare. Minimizing the risk of bias is crucial and mandates careful analysis and mitigation of biases in the training data and algorithms.
• Performance Metrics: Developing standardized metrics to evaluate SaMD performance is complex. An essential yet often-overlooked metric is generalization, measuring the SaMD’s effectiveness and safety across diverse patient populations.
• Continuous Learning: AI/ML SaMDs can evolve and adapt over time, in an unsupervised manner, as they learn from new data. Regulating these continuously learning systems to ensure they remain safe and effective is a significant challenge for many reasons, not the least of which is the tremendous burden that frequent regulatory review would create.
• Post-Market Monitoring: Ongoing monitoring of AI/ML SaMD performance after market entry is essential so that manufacturers and regulators can quickly identify and address safety and effectiveness issues as (or even before) they arise.
• Regulatory Consistency: Navigating different regulatory requirements across regions will be challenging for manufacturers. For example, the EU AI Act introduces additional complexities compared to the FDA medical device regulations.

The good news is that the FDA is ahead of the curve, actively working on advancing the regulatory science for AI/ML SaMDs. The FDA’s first effort came in their 2019 proposed AI/ML SaMD regulatory framework.4 Although this framework was originally intended to address only adaptive devices (i.e., algorithms that modified themselves on the fly as they gained experience and new data), the thinking has since been applied much more broadly to fixed AI/ML SaMDs as well.

This shift was acknowledged in the FDA’s 2021 Action Plan, which outlined a multipronged approach to developing regulations for SaMDs. The Plan included crafting a tailored regulatory framework specifically for AI/ML SaMD, including:

• Developing and requiring the use of Good Machine Learning Practices (GMLP) in design and testing.
• Using a patient-centered approach that encourages transparency.
• Creating new regulatory science methods that minimize algorithm bias and ensure robustness.
• Developing a real-work performance methodology that adopts a total product lifecycle approach to SaMD oversight.5

The FDA’s first efforts along the Plan’s outline came in 2023, when the FDA published a draft guidance document describing its desired contents for premarket applications that contain a Predetermined Change Control Plan (PCCP).6 The PCCP addresses the “continuous learning” point, where an adaptive AI/ML SaMD changes on its own through experience and retraining. Under this scenario, the FDA will review a proposed PCCP, as part of the whole premarket submission package, that describes in detail how an AI/ML SaMD will change over time and how the manufacturer will validate its continued safe and effective function. If the FDA agrees to the PCCP, then the manufacturer is free to make those changes without returning to the FDA for additional regulatory reviews. (Of course, if the FDA rejects the PCCP, then every significant change will require review prior to implementation.)

It is easy to see how the PCCP is a huge leap forward in regulatory science. Maybe someday PCCPs will become even more widely used than just for SaMDs. Who would oppose a process that ensures continued safety and effectiveness over a device’s lifecycle while reducing regulatory burden on manufacturers and regulators?

All AI/ML SaMDs share these three new, unique, and very important aspects that must be considered and understood when bringing a SaMD product to market that is safe and effective: Good Machine Learning Practices (GMLP), clinical trials and clinical validation, and the remarkable opportunity to use real-world evidence.

 

Good Machine Learning Practices (GLMP)

GMLP are essential for developing reliable, effective, and safe SaMD. Here are some key principles of GMLP:

• Data Management: Ensure high-quality, diverse, and representative data. Proper data cleaning, labeling, and augmentation are crucial.
• Feature Engineering: Extract meaningful features from the data. This involves selecting the right variables that will help the model make accurate predictions.
• Model Training and Evaluation: Use robust methods to train and evaluate models. This includes cross-validation, hyperparameter tuning, and ensuring the model generalizes well to new data.
• Bias and Fairness: Identify and mitigate biases in the data and model. This helps ensure the AI system is fair and does not disproportionately affect certain groups.
• Transparency and Explainability: Develop models that are interpretable and provide clear explanations for their decisions. This is important for gaining trust and ensuring accountability.
• Continuous Monitoring and Maintenance: Regularly monitor the model’s performance and update it as needed. This includes retraining the model with new data to maintain its accuracy and relevance.
• Regulatory Compliance: Adhere to relevant regulations and guidelines, such as those provided by the FDA for medical devices.

These practices help ensure that machine learning models are reliable, effective, and safe for their intended applications.

 

Clinical Trials

Clinical trials for AI-powered medical devices differ from traditional medical device trials in several key aspects:

• Data Sources: AI device trials often rely on retrospective data, meaning they test the algorithm on historical cases rather than live patients. This contrasts with traditional trials, which typically involve prospective data collection.
• Trial Design: Traditional medical device trials may use randomized controlled trials (RCTs) with blinding and placebos, though these are less common for device trials compared to pharmaceuticals. AI device trials might not follow this model strictly, focusing instead on the algorithm’s performance metrics.
• Adaptability: AI devices can evolve over time through continuous learning. This requires ongoing validation and monitoring, unlike traditional devices which are typically static once approved.
• Endpoints and Metrics: The endpoints for AI device trials can be more complex and varied, focusing on the algorithm’s accuracy, sensitivity, and specificity. Traditional device trials might have more straightforward clinical endpoints.
• Regulatory Pathways: The regulatory approval process for AI devices can be more iterative, with frequent updates and modifications to the algorithm. Traditional devices usually follow a more linear approval pathway.

 

Clinical Validation

It is important when designing a clinical validation (pivotal) study that these additional criteria be included:

• Safety and Effectiveness: Any clinical study for a SaMD must show that the device performs safely and effectively in real-world clinical settings according to its intended use. This involves rigorous testing to confirm that the device meets predefined performance standards and delivers accurate results.
• Generalizability: The clinical study must evaluate whether the AI/ML SaMD can generalize its outputs across different relevant patient populations and clinical environments.
• Bias Detection: The trial should be designed to identify and mitigate biases in the SaMD algorithm. This can be accomplished by testing the device on varied datasets, ensuring that it does not disproportionately affect certain patient groups.
• Post-Market Surveillance: Ongoing clinical validation after market introduction is important, and because SaMDs process data it is not difficult to stand up continuous monitoring methods that help detect issues that may arise.

These aspects are new additions to the concept of clinical validation and will go far toward maintaining high standards for AI/ML-powered medical devices.

 

Real-World Evidence

Finally, SaMDs are uniquely positioned to monitor their own performance in the real world after market introduction. Because SaMDs intake and output data, device use will by necessity create large datasets that can be analyzed post hoc. This real-world evidence (RWE) will play a significant role in continuing validation, feedback into product design and improvement, and potential of expansion of the indications for use. Here are some key aspects:

• Performance in Real-World Settings: RWE helps assess how SaMDs perform in everyday clinical environments, beyond controlled trial settings.
• Generalizability: By using data from a wide range of sources and patient populations, RWE helps determine if the SaMD will work well across different demographics and clinical conditions.
• Bias Detection and Mitigation: RWE can reveal biases that might not be apparent in controlled premarket trials. This discovery can feed back into product design, correcting biases and ensuring fair and equitable device performance.
• Continuous Learning and Improvement: RWE provides the ongoing data needed to monitor and improve these devices over time, ensuring they remain as good as (or even improve) in accuracy and reliability.
• Regulatory Support: The FDA is increasingly accepting of RWE to support the approval and post-market surveillance of SaMDs.

These aspects highlight the importance of RWE in ensuring that AI-powered medical devices are both safe and effective for widespread clinical use.

 

Frequently Asked Questions (FAQ)

NAMSA experts stand ready to assist you in all phases of your SaMD product development. Feel free to call upon us for a consultation. Here are a few questions that we often hear from clients at different stages in their journey.

 

Is my software a medical device?

It’s not always clear whether your software meets the definition of a medical device, particularly if it is identifying the presence of an abnormal condition (such as heart failure or a bacterium). NAMSA’s experts can work with you to formulate an appropriate IFU (indications for use) statement and follow that with a landscape survey of the applicable device space. If it’s still not clear, NAMSA can work with you and FDA to reach a final, binding designation for your product.

 

How would I go about collecting clinical evidence that my SaMD is safe and effective?

SaMD evaluations are excellent examples of how NAMSA can work with you to design creative approaches to clinical evidence generation. For example, there may be opportunities to use retrospective data in a prospective study that would test your device’s performance. Or there may be opportunities to combine premarket and post-market data collection efforts. It is often unnecessary to take the traditional path of a randomized, controlled, prospective trial for SaMD evaluations.

 

What exactly is a PCCP and why would I want one?

A predetermined change control plan (PCCP) describes in detail exactly what change(s) you intend to make to your SaMD, exactly how and when you will execute those changes, and how you will validate your SaMD’s continued safety and effectiveness. Although PCCPs seem like a lot of work, NAMSA’s experts can help you get to an FDA-reviewed and accepted predetermined change control plan that will be a huge boon. It can:

• Help you avoid having to return to FDA for repeated reviews after each significant change, freeing you to continually improve your SaMD’s performance over time.
• Reassure your company’s investors and directors that you have a well-defined path forward for your product.
• Raise awareness of your company and your product because as of this writing PCCPs are still a very new and novel thing, showing how you are ahead of your competition and thinking forward.

REFERENCES

1. The Congress of the United States of America. Federal Food, Drug, and Cosmetic Act. Published online September 30, 2023. Accessed June 26, 2024. https://www.govinfo.gov/content/pkg/COMPS-973/pdf/COMPS-973.pdf
2. Center for Devices and Radiological Health. Software as a Medical Device (SaMD). FDA. December 4, 2018. Accessed August 26, 2024. https://www.fda.gov/medical-devices/digital-health-center-excellence/software-medical-device-samd
3. IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a Medical Device”: Possible Framework for Risk Categorization and Corresponding Considerations. Published online 2014:30.
4. Center for Devices and Radiological Health. Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback. Published online 2019. Accessed October 16, 2023. https://www.fda.gov/media/122535/download?attachment
5. Center for Devices and Radiological Health. Artificial Intelligence and Machine Learning in Software as a Medical Device. FDA. September 22, 2021. Accessed September 21, 2023. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
6. Center for Devices and Radiological Health. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions. March 30, 2023. Accessed August 21, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial