Microsoft Diagnostic AI Tops Generalist Doctors in Science Study

What happened

Microsoft's medical AI research has moved from preprint to peer review. On Thursday, the journal Science published "Sequential Diagnosis with Language Models," the paper behind the Microsoft AI Diagnostic Orchestrator, or MAI-DxO. The paper first appeared on arXiv last year without peer review.1 The headline finding has not changed. On a set of hard published cases, the system was more than twice as accurate as practicing generalist physicians, and it ordered fewer tests while doing so.12

The figures match across outlets. On 304 difficult case reports from The New England Journal of Medicine, MAI-DxO running on OpenAI's GPT-5 reached the correct diagnosis 83.2% of the time. Generalist doctors who could use textbooks and the internet scored 36.1%.10 Without reference materials, the doctors' accuracy fell to roughly 22%, which puts the AI close to four times ahead in that setup.16 On a second set of 325 emergency department admissions, the gap was smaller but still large: 80.3% for the system against 46.8% for the physicians.10

Microsoft says real patients come next. Dr. Dominic King, the company's health AI lead, said testing will begin at Beth Israel Deaconess Medical Center in Boston within the next couple of months.1 Other coverage describes the next phase as an evaluation in emergency department waiting rooms.35 One report says the AI's recommendations will be checked against what clinicians actually did.4 Microsoft has not given a timeline for selling the product.6

Why the method matters more than the score

The accuracy numbers get the headlines, but the bigger change is in how the test was run. Most medical AI systems look at a patient's symptoms, lab results and imaging all at once and produce an answer in one pass.2 MAI-DxO starts with only a chief complaint. It then has to decide which questions to ask and which tests to order before it commits to a diagnosis.1 Inside the system, several AI agents play different roles: some propose diagnoses, some choose tests, some challenge assumptions and some track costs. The researchers argue this setup can counter biases that trip up individual doctors, such as anchoring on a first impression.6

King put the case against older benchmarks simply. He said AI now passes multiple-choice medical exams almost perfectly, but real patients never arrive with a tidy summary.10 Harsha Nori, head of AI at Microsoft AI Health, said the team scored how carefully and efficiently the system worked through a case, not just whether it got the answer right.9 Microsoft has packaged the cases into a benchmark called SDBench, which it plans to open-source.6

The benchmark may matter more than the product. If SDBench catches on, every medical AI developer would have to show its system can conduct an investigation, not just recognize patterns. That is a higher bar, and arguably a more honest one.

Where the coverage diverges

Outlets agree on the numbers but frame them differently. India Today leans on Microsoft AI chief Mustafa Suleyman's claim that this is a step toward "medical superintelligence."4 Suleyman has predicted performance indistinguishable from humans within a couple of years, while also admitting the system has only been tested on past cases and still needs clinical validation.6 Firstpost and Gizmodo are more cautious. Firstpost says the results do not show the tool is ready to replace doctors or how it would handle the full range of hospital patients.5 Gizmodo notes that experts see these tools as assistants, not substitutes.1

A detail many reports skip is how much of the gain comes from the underlying model rather than from Microsoft's orchestration layer. According to India Today, GPT-5.5 inside the orchestrator raised accuracy on complex cases from 72% to nearly 87%. A weaker model gained even more: Google's Gemini 2.5 Flash went from 46.4% alone to almost 69% inside the system. But on the faster emergency cases, the orchestrator added little accuracy or cost benefit for the most advanced models.4 So the ER comparison against doctors largely reflects how strong the frontier models already are, not Microsoft's architecture. That distinction gets lost when the results are reduced to "AI beats doctors."

The caveats that should temper the headline

The study has several limits that matter. First, the doctors were generalists, not specialists, and the researchers themselves expect the gap to narrow against experts.1 Second, the claimed savings of half to a third of the cost are estimates from simulated cases, not real medical bills.1 Third, every case was presented as text. Neither the AI nor the doctors could examine a patient or look at medical images.10

That last limit matters a lot for radiology, the field where clinical AI is most established. MAI-DxO worked from text descriptions of tests. It did not read scans. Imaging is where regulated AI already lives, as the FDA's records show, so this study does not test the system against the kind of AI hospitals already buy.

There are also human factors. A 2025 study found that doctors who used AI to flag possible cancer risks during colonoscopies performed worse once the tool was taken away.1 Accountability is another open question. Even a successful trial would not settle who is responsible when an AI-assisted diagnosis goes wrong.6

The regulatory gap

This is where the story gets hard for Microsoft. As of September 2026, the FDA says it has authorized more than 1,600 AI-enabled medical devices for the U.S. market.11 Its examples are narrow, single-purpose tools: imaging systems that flag skin cancer, software that detects diabetic retinopathy from retinal photos, and algorithms that adjust insulin doses.11 Authorizations have climbed fast, from six in 2015 to 335 in 2025, with 181 more in the first half of 2026.18

Radiology dominates. In one count of 1,451 listed devices, 1,104 were radiology tools, far ahead of cardiovascular devices at 141.12 The most recent entries look the same: mammography readers, lung nodule detectors, CT analysis software and ultrasound systems. Nearly all reached the market through the 510(k) pathway, which only requires showing a device is substantially equivalent to one already on the market.14

MAI-DxO fits none of those categories. The FDA had not authorized any device built on generative AI or large language models as of March 2026.12 The Congressional Research Service reported the same in June. It noted that the FDA had given a breakthrough designation to a patient-facing generative AI app from RecovryAI and had discussed how to regulate such systems at its Digital Health Advisory Committee.19 Breakthrough designation speeds review, but it is not marketing authorization. Designated devices still have to meet the FDA's safety and effectiveness standards.13

Put together, the picture is clear. Today's regulatory system was built for narrow tools that do one well-defined task, usually on images. A general-purpose diagnostic system that decides which tests to order across every specialty is a different kind of product. It also runs on frontier models that change every few months: this study's best results came from GPT-5, with more gains on GPT-5.5.1 It is not clear how regulators would review a device whose core engine keeps changing.

The bottom line

This study does not mean AI can replace doctors, and it does not mean Microsoft's tool is close to approval. Its real value is that it changes the test. Peer-reviewed evidence that an AI can work through a diagnosis step by step, more cheaply and more accurately than generalists, is a significant result. The case for asking doctors to treat AI as a working partner in diagnosis is now much stronger.

But a benchmark only gets the system to the hospital door. The Beth Israel Deaconess trial will show whether the gains hold up with real, messy patients. The harder question is whether the FDA can build a pathway for general-purpose diagnostic AI before tools like this reach clinics through less formal routes. Microsoft already sells digital tools to U.S. hospitals, and health questions are among the most common uses of its consumer Copilot chatbot.6 Patients are already asking AI about their health. The question is whether regulators can approve a clinical-grade version before that informal use becomes the norm.