Approved Without Proof

Over 1,250 AI-enabled medical devices have been authorized for marketing in the United States, but a cross-sectional study found that only 55.9% had publicly available clinical performance data at the time of clearance. Most were cleared through the 510(k) pathway based on bench testing or retrospective dataset performance, without prospective clinical validation demonstrating real-world diagnostic accuracy. Among devices that did undergo pivotal studies, one-third of De Novo devices failed to meet their primary effectiveness endpoints but were still authorized. Clinicians integrating AI tools into diagnostic and treatment workflows cannot independently assess whether the device performs as claimed in their specific patient population, clinical setting, and workflow context.

AI-enabled devices are used across radiology (stroke detection, pulmonary embolism triage, fracture detection), cardiology (ECG interpretation), pathology (cancer detection), and ophthalmology (diabetic retinopathy screening) — specialties where clinicians make time-critical decisions based on AI outputs. When independent academic validation studies have been conducted, they frequently find performance below manufacturer claims in real-world settings. If an AI device has a higher false-positive rate in a specific demographic or clinical setting than its clearance data suggests, patients may undergo unnecessary interventions or experience delayed treatment for life-threatening conditions.

The 510(k) pathway requires demonstration of substantial equivalence to a predicate device, not independent clinical validation — and for AI devices, the predicate may use an entirely different underlying technology, making the comparison structurally weak. One-fifth of De Novo-authorized AI devices were never evaluated in pivotal studies at all. The FDA's AI/ML action plan and Predetermined Change Control Plan (PCCP) guidance address how algorithms can be updated post-market but do not address the baseline clinical validation gap. Some academic institutions have begun independent validation studies, but these require access to clinical datasets that raise privacy and cost barriers. Manufacturers consider performance data proprietary and competitive, actively resisting transparency mandates. No standardized reporting framework exists for AI device performance analogous to STARD for diagnostic accuracy studies, though TRIPOD+AI is emerging.

A regulatory mechanism that requires minimum clinical evidence thresholds for AI devices affecting clinical decision-making — without requiring the full PMA pathway that would stifle innovation — would close the gap. A standardized, publicly accessible performance reporting framework (building on TRIPOD+AI) that enables clinicians to compare AI devices by clinical setting, patient demographics, and workflow integration would transform purchasing and adoption decisions. Federated validation approaches, where performance is tested across institutional datasets without centralizing patient data, could resolve the privacy-versus-transparency tension.

A student team could build a systematic database of FDA-cleared AI medical devices and their publicly available clinical evidence, extending existing cross-sectional analyses and creating a tool clinicians could use to evaluate devices before adoption. A team with machine learning and clinical skills could conduct an independent validation study of one or more FDA-cleared AI diagnostic devices using institutional or publicly available imaging datasets, documenting performance across demographic subgroups. A design-oriented team could prototype a "nutrition label" format for AI device performance that presents accuracy, sensitivity, specificity, and demographic breakdown in a standardized, clinician-friendly format.