Source: PLOS Digital Health — 2026-08-19
Summary
A systematic analysis of every AI/ML-enabled medical device cleared or approved by the FDA through December 5, 2025 — 1,357 devices in total — found that only 3 of them (0.2%) were ever evaluated for their effect on patient-centered outcomes like mortality, morbidity, or hospital readmissions. Researchers led by Rawan Abulibdeh at the University of Toronto cross-referenced the FDA device database and the ACR Data Science Institute catalogue against ClinicalTrials.gov and PubMed, and found that regulatory clearance has dramatically outpaced clinical validation.
Key Takeaways
- Of 1,357 FDA-cleared AI devices, just 34 (2.5%) are linked to a registered prospective trial, only 12 (0.9%) have posted trial results, and only 12 (0.9%) have a peer-reviewed publication behind them.
- Only 3 devices (0.2%) were ever tested against actual patient outcomes — mortality, morbidity, readmissions — rather than a technical accuracy proxy.
- Radiology accounts for roughly 78% of all cleared AI devices (1,059 of 1,357), followed by cardiovascular (9%) and neurology (5%) — meaning the imaging specialty with the most AI tools also has the thinnest outcomes evidence base.
- Most devices reach market via the 510(k) pathway by showing "substantial equivalence" to an existing predicate device — a bar that requires no demonstration that the tool actually improves what happens to a patient.
- The authors call the situation a "profound validation gap," arguing evidence standards need to catch up with the pace of clearance before AI outcome claims can be trusted at the bedside.
Reel Script
Hook (~15-20s): Your hospital's radiology AI was cleared by the FDA. That does not mean anyone checked whether it actually helped a single patient survive, recover faster, or avoid readmission. Only 3 devices out of 1,357 ever were.
Core Concept (~45-90s): Most AI medical devices get to market through the FDA's 510(k) pathway, which only requires showing "substantial equivalence" to a device already on the market — essentially, proving the new tool works about as well as an old one, not that it improves a patient's actual health. A device can clear that bar on technical accuracy alone, like matching a radiologist's read rate in a lab setting, without ever being tested in a real hospital to see if it changes what happens to the person on the table. Researchers at the University of Toronto systematically checked every one of the 1,357 AI devices the FDA has cleared against two things: registered clinical trials on ClinicalTrials.gov, and published results in PubMed.
Hands-On (~45-150s): The numbers tell the story. Of 1,357 cleared devices, 34 (2.5%) are tied to a registered prospective trial at all. Of those, only 12 (0.9%) ever posted results, and only 12 (0.9%) produced a peer-reviewed paper. And the outcome that matters most to patients — did this change mortality, complication rates, or hospital readmissions — was measured in just 3 devices out of the entire 1,357. Radiology, which holds 78% of all cleared AI devices, is simultaneously the specialty with the least outcomes testing relative to its device count, since most radiology AI clearances rely on matching predicate performance rather than patient-level trials.
Takeaway (~20-30s): "FDA-cleared" is not the same claim as "clinically proven to help you" — right now it almost never is, and hospitals adopting these tools are largely taking it on faith. If you're evaluating clinical AI for procurement, demand the outcomes trial, not just the clearance letter.