health · digital · family: the rulebook was written for a world that no longer exists
the algorithm moved on, the approval didn't
FDA's static approval framework cannot keep pace with continuously learning AI medical devices
Problem statement
Traditional FDA regulatory pathways (510(k), De Novo, PMA) were designed for medical devices that are developed, tested, and released in a fixed form. AI/ML-enabled device software functions can improve with new data — the FDA calls this one of their greatest potential benefits — but a modification that could significantly affect safety or effectiveness generally requires a new premarket submission unless it was specified in an authorized Predetermined Change Control Plan. This creates a fundamental mismatch: the regulatory framework penalizes the very property (continuous learning) that makes AI/ML devices valuable. Over 1,250 AI-enabled devices were authorized for marketing in the United States as of July 2025, and the oversight mechanisms now in place do not amount to a comprehensive framework for monitoring them in real-world use.
Why this matters
Over 1,250 AI-enabled devices span radiology, cardiology, ophthalmology, pathology, and other specialties, many used in time-critical diagnostic workflows such as stroke detection and pulmonary embolism triage. Delays in deploying algorithm improvements — due to regulatory bottlenecks where review timelines sometimes exceed one year — directly translate to diagnostic performance worse than the manufacturer's current best model. Conversely, deploying unvalidated updates risks patient harm, and the regulatory uncertainty has been cited as a significant barrier to investment in AI medical device development.
What’s been tried and why it hasn’t worked
The FDA's Predetermined Change Control Plan (PCCP) framework, finalized in December 2024, allows manufacturers to pre-specify what algorithm changes they plan to make, the methodology for making those changes, and the assessment protocols. However, manufacturers report that defining change boundaries prospectively is extremely difficult for genuinely adaptive algorithms — no validated methodology exists for predicting the performance envelope of a retrained model before retraining occurs. The 510(k) pathway, used for the majority of AI device clearances, assesses substantial equivalence to a predicate at a fixed point in time; once the algorithm updates, that equivalence determination may no longer hold, but there is no systematic mechanism to re-evaluate it. Existing quality management system frameworks (design controls, risk management) were built for hardware-centric manufacturing and require substantial adaptation for software that changes continuously.
What would unlock progress
A validated technical methodology for bounding the performance envelope of retrained ML models — essentially, a way to guarantee that a model update stays within defined safety and efficacy parameters without requiring full de novo clinical validation each time — would directly address the core regulatory bottleneck. Standardized automated testing suites and benchmark datasets that could serve as "regression tests" for clinical AI performance after retraining would enable both manufacturers and regulators to assess updates rapidly. Regulatory science research bridging ML theory (distributional shift, model drift) and clinical validation practice could create the intellectual foundation for a new lifecycle-based approval paradigm.
Entry points for student teams
A student team could develop a model monitoring framework that detects algorithmic drift by tracking performance metrics on curated benchmark datasets over time, creating an early-warning system for when a retrained model deviates from its approved performance envelope. Another entry point would be prototyping a "regression test suite" for a specific clinical AI application (e.g., chest X-ray triage) that demonstrates how continuous validation could work in practice. Teams with backgrounds in machine learning, software engineering, and regulatory science would be well-suited. A scoped semester project could use publicly available clinical AI models and research datasets rather than requiring proprietary device access: CheXpert, which its maintainers describe as a public dataset (downloaded through the Stanford AIMI site under its terms and conditions), or MIMIC, which PhysioNet restricts to credentialed users who complete required training and sign a data use agreement.
Genes
Structural cousins: stuck the same way, in other fields
Sources
FDA Final Guidance, "Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions," FDA CDRH, accessed 2026-02-19 go to source ↗
Working notes
Notes made while writing and checking this brief, shown as written. They use working shorthand.
Key references include the FDA PCCP Final Guidance (December 2024), FDA Draft Guidance on AI-Enabled Device Software Functions (January 2025, https://www.fda.gov/media/184856/download), Bipartisan Policy Center analysis of FDA oversight of health AI tools (November 2025), and The Lancet Digital Health analysis of FDA-cleared AI/ML device predicate networks (2023). The January 2025 draft guidance encourages sponsors to consider a PCCP and describes performance monitoring plans that identify and respond to performance changes after deployment, including data drift; under the PCCP final guidance, significant modifications not specified in an authorized plan likely require a new marketing submission. The Bipartisan Policy Center brief says critical questions remain about how to define intended use for adaptive or generative algorithms and how to ensure consistent performance after deployment. This brief is closely related to digital-ml-component-formal-verification and digital-ai-trustworthiness-heterogeneous-verification, which address analogous verification challenges outside the medical domain. Tagged as "worsening" because the number of AI-enabled devices on the market is growing rapidly while the regulatory framework lags further behind.
Reconciliation 2026-09-30: Whole write-up read against the four sources it names; all fetched 2026-09-30, and a second reader went over the repairs the same day, opening every source again. Read in full: the FDA final guidance on Predetermined Change Control Plans (guidance page https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence and its PDF https://www.fda.gov/media/166704/download, the version issued August 18, 2025, which states it was originally issued December 4, 2024); the FDA draft guidance of January 7, 2025 (https://www.fda.gov/media/184856/download); the Bipartisan Policy Center issue brief "FDA Oversight: Understanding the Regulation of Health AI Tools" by Maya Sandalow, Katie Adams and Gabriel Loud, dated November 10, 2025 (the publisher's page refuses automated requests, so it was read from the Internet Archive copy of 2026-07-13, https://web.archive.org/web/20260713052343/https://bipartisanpolicy.org/issue-brief/fda-oversight-understanding-the-regulation-of-health-ai-tools/); and the predicate-network paper, Muehlematter, Bluethgen and Vokinger, "FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks," The Lancet Digital Health 5(9):e618-e626, 2023 (citation from the Crossref record https://api.crossref.org/works/10.1016/s2589-7500(23)00126-7 and Europe PMC record 37625896; the publisher and the Zurich repository refuse automated access, so the full text was read from the Internet Archive copy of the repository PDF, https://web.archive.org/web/2024id_/https://www.zora.uzh.ch/id/eprint/238337/1/ZORA_238337.pdf). Changes: (1) The text said each significant modification "currently requires a new premarket submission" and that AI device software is "inherently adaptive." The final guidance says modifications specified in an authorized plan can be made without a new marketing submission, and that updates can range from a device that is primarily "locked" to continuous ones; it calls the ability to improve by learning from real-world data one of the greatest potential benefits of AI and ML. The sentence now says that. (2) The text said over 1,250 devices are on the market "with no standardized approach to lifecycle management of algorithm updates." The Bipartisan Policy Center brief says the FDA's public database lists over 1,250 AI-enabled devices authorized for marketing as of July 2025, and that existing oversight mechanisms "do not constitute a comprehensive framework for monitoring AI-enabled devices in real-world settings"; the sentence now follows that wording. (3) The sentence that the Total Product Life Cycle Advisory Program enrolled 63 Breakthrough Devices in 2024 was removed. None of the four sources mentions the program. (4) Entry points: the text called CheXpert and MIMIC "publicly available." The MIMIC pages on PhysioNet (https://physionet.org/content/mimic-cxr/ and https://physionet.org/content/mimiciv/) say only credentialed users who complete required training and sign a data use agreement can access the files, and the sentence now says so in those words. CheXpert's own page (https://stanfordmlgroup.github.io/competitions/chexpert/) calls it a large public dataset and sends downloads to the Stanford AIMI site; it is kept as the open door. (5) Source Notes dated the Bipartisan Policy Center analysis 2024; the page is dated November 10, 2025, and its 1,250 figure is for July 2025. The date now reads November 2025. (6) Source Notes placed the predicate-network analysis in "Nature Digital Medicine." The Crossref and Europe PMC records and the repository copy place it in The Lancet Digital Health, and a Crossref search found no such paper in npj Digital Medicine. The journal name was corrected. (7) Source Notes said the January 2025 draft guidance "left unclear how to handle emergent algorithmic behaviors that fall outside predefined change envelopes — a gap that remains unresolved." The draft does not use those terms; it encourages sponsors to consider a change control plan and describes performance monitoring plans that identify and respond to performance changes after deployment, including data drift. The final guidance says significant modifications not specified in an authorized plan likely require a new marketing submission, and the Bipartisan Policy Center brief says critical questions remain about defining intended use for adaptive or generative algorithms and ensuring consistent performance after deployment. The sentence now says those three things. (8) On the Source line, the guidance title was corrected to the title on the FDA page ("Artificial Intelligence-Enabled," not "AI-Enabled"). The link on the Source line is unchanged; it opens the FDA guidance search page, and the guidance's own page is the first address in this note. Confirmed in the sources and left as written: the pathways were built for static devices (Bipartisan Policy Center); the plan framework was finalized in December 2024 and lets a manufacturer describe planned modifications, the method for making them and an impact assessment (FDA final guidance); the over-1,250 count, up from 950 in August 2024 (Bipartisan Policy Center); the 510(k) pathway rests on substantial equivalence to a predicate and the majority of AI/ML-based devices have been cleared through it (Lancet Digital Health paper); radiology, cardiovascular and ophthalmology among the specialties, and the short time window for detecting large vessel occlusion of the brain (Lancet Digital Health paper); the January 2025 date of the draft guidance. Could not be checked, because they are in none of the four sources; left as written: pathology among the specialties and the pulmonary embolism example; review timelines that "sometimes exceed one year"; regulatory uncertainty as a barrier to investment; that manufacturers report change boundaries are extremely difficult to define and that no validated methodology exists; that there is no systematic mechanism to re-evaluate equivalence after an update; and that quality system frameworks were built for hardware-centric manufacturing. The Bipartisan Policy Center brief comes closest on two of these: it says some developers find the premarket pathways unclear and time-intensive and that manufacturers have discontinued tools citing the cost and complexity of authorization, and that the agency has limited mechanisms to track real-world performance and model drift. Titles unchanged. Genome tags untouched.