FDA Weighs Doctor-Style Testing for AI Medical Devices

A New Model for Evaluating Medical AI

The Food and Drug Administration is exploring a fundamentally different way to evaluate generative AI-enabled medical devices: testing them the way it would test a human clinician. Rather than relying solely on traditional engineering-style validation, the agency is weighing a "competency-based approach" that would assess whether an AI system can reason through clinical scenarios and reach sound medical judgments, much as a doctor's competency is assessed through training and examination 18. The idea, laid out in an agency discussion document, reflects growing recognition that generative AI tools behave less like static software and more like adaptive decision-makers whose performance can shift with new data and updates 8.

A Regulatory Framework Playing Catch-Up

This proposal is the latest sign that the FDA is rethinking its oversight structure as breakthrough AI products flood into the healthcare market. The agency has already granted breakthrough status and clearances to novel tools such as PanCancer Detect, part of a broader push to accelerate promising cancer-detection technology while simultaneously tightening scrutiny and adding new evidentiary requirements for AI-driven diagnostics 25. The scale of this shift is notable: at least 1,357 AI-enabled medical devices have now been authorized by the FDA, roughly double the number cleared through 2022, underscoring how quickly the technology is being integrated into clinical practice 7.

Real-World Devices, Real-World Risks

The push for smarter regulation comes as reports surface of AI systems misidentifying body parts and introducing new risks during surgical procedures, raising questions about how well current oversight catches errors before they reach patients 7. Cybersecurity adds another layer of concern; experts warn that AI-enabled devices, if not built with security as a foundational principle, could become dangerous points of failure in high-stakes clinical settings 3. At the same time, the FDA continues to greenlight ambitious new hardware, including a first-of-its-kind automated blood draw device intended for outpatient use, signaling the agency's willingness to approve automation even in physically invasive procedures 6.

Consumer Wearables Complicate the Picture

The tension between innovation and regulatory caution is also playing out in consumer technology. WHOOP has said it will keep a controversial blood pressure feature in its new WHOOP MG device despite pushback from the FDA, a standoff that could shape how wearables offering ECG scans, AI coaching, and blood pressure estimates are regulated going forward 4. That dispute highlights a broader gray area between medical-grade devices subject to strict FDA clearance and consumer wellness products that increasingly blur the line into diagnostic territory.

Why It Matters

Taken together, these developments suggest the FDA is trying to build a more flexible, judgment-based regulatory model suited to AI systems that learn and adapt rather than behave like fixed instruments. How the agency balances rapid approvals of breakthrough tools against documented safety failures, cybersecurity vulnerabilities, and disputes with device makers will likely determine public trust in AI-driven diagnosis and treatment for years to come.