Hospital AI Review Maps Four Gaps in US Medical AI Oversight

A review that counts the cost of speed

A systematic review published this month in Frontiers in Digital Health finds that artificial intelligence in American hospitals is spreading faster than the rules meant to govern it. It goes further than most commentary on the subject because it identifies specific, measurable gaps in oversight. The review was led by Joseph Conteh of the University of New Mexico with co-authors Naim Kapucu, Eunice F. Ebevuhe and Oluwapelumi Wumi Adeniji. The team screened 1,065 records published between January 2017 and January 2026 and kept 22 studies, then ran a structured gap analysis of federal statutes, regulations and agency guidance21. Its conclusion is that U.S. oversight of hospital AI is fragmented and reactive, even as the FDA's count of authorized AI-enabled devices passed 1,451 by the end of 202522.

The paper landed during a week of similar warnings. Politico reported that health systems are adopting AI agents for record-gathering, patient messaging, charting and scheduling faster than they can supervise them23. A Harvard Medical School team described physician uptake of AI, in a study published in JAMA, as faster than almost any earlier clinical technology. The researchers contrasted it with electronic health records, which spread slowly and mostly after government mandates24. A ScienceSoft industry report for the third quarter of 2026 found that deployment is widespread but that consistent ways of proving clinical or economic value are mostly missing27. These sources all describe the same pattern: hospitals keep adopting AI while the evidence and the rules lag behind.

The device count keeps climbing, and radiology dominates it

The figure in the review is already out of date. The FDA's September 2026 update of its AI-enabled device list contains 1,614 authorizations through the end of June 2026, 5.9% more than the previous quarter19. Of those, 1,230, or about 76%, are radiology devices. The agency authorized 89 AI devices in the second quarter of 2026 and 92 in the first19. GE HealthCare leads manufacturers with 134 authorizations, followed by Siemens Healthineers with 101 and Philips with 62, and those totals include acquired companies19.

Radiology's share has stayed between roughly 71.5% and 80% of annual clearances in every year tracked11. In 2025 the single most common product code was radiological computer-assisted detection and diagnosis, which accounted for 75 of 295 clearances that year11. Most AI devices reach the market through the 510(k) pathway, which requires a showing of substantial equivalence to an existing device rather than new clinical trials17. The technology is also changing. Aidoc received clearance in January 2026 for what has been described as the first foundation-model-powered clinical AI device, a body CT triage tool that covers 14 conditions18. As of March 2026, however, the FDA had not authorized any device that uses generative AI or large language models20.

In short, the regulated portion of hospital AI is mostly imaging software reviewed against older devices. The newest and most general-purpose tools have barely entered the device framework.

Strong benchmarks, weaker results on the ward

The review's central technical finding is the gap between how well a tool performs before approval and how well it performs once deployed. One COVID-19 chest X-ray model scored an AUC above 0.95 on open datasets. After prospective rollout across several hospitals it fell to about 0.70, and it ranged from 0.66 to 0.74 over 19 weeks21. Readmission models with AUCs of 0.96 and 0.83 had been tested only on retrospective data with no external validation21. The authors make a distinction worth keeping: many of these weaknesses come from the research behind FDA submissions, not from the FDA's statutory standards. The two problems need different fixes21.

The review also reports clear successes. An FDA-authorized pulmonary embolism tool reached 99.9% negative predictive value across 11,736 real-world scans. It cut worklist detection time from 7,714 minutes to 87 and reduced missed cases from 44.8% to 2.6%21. A sepsis algorithm was linked to a fall in in-hospital mortality from 21.3% to 9.0%21. The authors call such results exceptions rather than the norm. Across 70 CMS quality indicators from 2022 to 2025, hospitals using AI did better on some measures and worse on others, including readmissions and sepsis treatment compliance. Those findings are observational and do not show cause and effect21.

Four gaps that reinforce each other

The gap analysis names four connected failures[22]:

• Post-market monitoring. In a national sample, only 58.1% of hospitals monitored AI after implementation in 2024, and most of those did not check continuously for drift21.
• Equity assurance. One readmission model was trained on data that was less than 2% Hispanic and Asian and more than 75% white, and it had no subgroup testing. A triage system was 11.4 percentage points less accurate for children than for adults21.
• Accountability. Legal liability, clinical responsibility, institutional governance and vendor obligations are often treated as one question. This matters more because 79% of hospitals use predictive models built by their electronic health record vendors, and no framework assigns responsibility among hospitals, developers, vendors and clinicians21.
• Governance capacity. Larger, health-system-owned and more profitable hospitals adopt and audit AI at higher rates. Facilities in shortage areas and deprived regions are falling behind21.

The accountability finding matches what malpractice specialists report. Liability currently tends to land on providers rather than manufacturers, and physicians' use of AI rose from 38% to 66% in a single year of AMA survey data28.

The data on hospital monitoring conflict

The monitoring figures do not agree from source to source. Federal data from ASTP/ONC, published in September 2025, show that 82% of hospitals using predictive AI evaluated models for accuracy, 74% for bias and 79% monitored them after deployment25. The review's figures are lower: 58.1% doing post-implementation monitoring and, in another dataset, only 44% assessing bias21. The difference probably comes from different denominators and definitions. The federal figures cover hospitals that already use predictive AI and count any evaluation, while the review stresses that most monitoring is sporadic rather than continuous21.

Industry surveys support the lower picture. A Black Book survey of 182 hospital leaders found that only 29% had enforced policies covering AI model inventory and approvals. Only 22% were confident they could produce a full, auditable explanation of an AI tool within 30 days if a regulator or payer asked26. An HFMA report found that 88% of health systems use AI but only 18% have mature governance26. The most likely explanation is that many hospitals can say they evaluate their models, but few have the continuous, documented oversight the review calls for.

Washington is moving toward less regulation

The review recommends lifecycle governance with six parts: rigorous pre-market validation, mandatory post-deployment drift monitoring, standardized transparency aligned with ONC's HTI-1 rule, equity audits, clear accountability, and support for under-resourced hospitals21. It holds up the EU AI Act as a contrast, since it requires conformity assessment for high-risk medical AI and places explicit obligations on deployers as well as providers21.

U.S. policy in 2026 is mostly moving the other way. On January 6 the FDA revised its clinical decision support guidance. Commissioner Marty Makary described the change as cutting unnecessary regulation3. The agency will now use enforcement discretion for tools that give a single clinically appropriate recommendation, as long as clinicians can review the reasoning behind it5. Law firm analyses noted that the guidance does not actually address AI, generative models or patient-facing chatbots3. The ACR stressed that any software analyzing medical images for diagnosis is still subject to FDA oversight, so radiology AI stays inside the device pathway1. One compliance tracker concluded that more decision-support tools can now reach clinicians without premarket review, which leaves hospitals to vet them. It also pointed to an ONC proposal, HTI-5, that would remove the AI "model card" transparency requirements9. That matters because the review treats HTI-1's transparency rules as a foundation to build on21.

There is one opening. In August the FDA published a discussion paper on regulating generative AI devices that sketches a risk map, competency testing and more post-market monitoring. It is not binding, and comments close on October 197. Makary has also said post-market monitoring should get more emphasis2.

What it adds up to

The authors acknowledge real limits to their evidence. Only 22 studies made the cut, many were retrospective and single-center, and the review protocol was registered after the fact21. Its main argument still holds when set against the other reporting from this month. Hospitals are taking on more responsibility for checking whether AI works, while federal agencies are reducing their own premarket role and possibly cutting transparency rules. Radiology AI remains under FDA review, but that review mostly happens before deployment, and the review shows performance can decline afterward.

The practical consequence is that well-funded health systems may be able to monitor their tools adequately, while rural and independent hospitals, which already use AI less, probably cannot25. Unless regulators fund monitoring capacity or require post-market surveillance, the review's warning about risks growing alongside deployment is likely to prove accurate.