AI Diagnosis Liability: Harvard ER Study Outpaces the Law

A model that held its own in the ER

A research team of physicians and computer scientists from Harvard Medical School and Beth Israel Deaconess Medical Center has published a study in Science testing how OpenAI's large language models perform across several medical tasks. The headline experiment used real emergency room cases 1. In it, the researchers took 76 patients who came through Beth Israel's emergency department. They compared diagnoses from two internal medicine attending physicians with diagnoses generated by OpenAI's o1 and 4o models 1. Two other attending physicians then graded the diagnoses without knowing which came from a human and which came from a machine 12.

The study reports that at every diagnostic touchpoint, o1 performed either nominally better than or on par with both the physicians and 4o. The gap was widest at the first touchpoint, when the least information was available 1. NPR described the reasoning model as matching and often beating both doctors and an earlier OpenAI model 4. It also illustrated the stakes with a case involving a patient whose pulmonary embolism symptoms worsened after initial improvement. The AI, drawing on the patient's records, proposed that a history of lupus might explain what was going on 4.

Coverage varies on model naming. One outlet refers to o1-preview 2, another to o1 and 4o 1, and NPR frames the comparison against GPT-4 4. These differences likely reflect shorthand rather than conflicting findings, but readers comparing reports should keep them in mind.

Why the messy data matters

What separates this work from earlier AI-in-medicine benchmarks is the input. According to one report, the researchers did not clean up the cases. They fed the models information as it appeared in the electronic health record 2. That matters because polished vignettes and multiple-choice exams have long flattered AI systems. Co-author Peter Brodeur, a Harvard clinical fellow at Beth Israel Deaconess, noted that models once evaluated with multiple-choice tests now score close to perfect on them 2. That saturation is the sense behind the "we're already at the ceiling" framing 2. Real charts are full of noise, abbreviations and gaps, so strong performance there is a more meaningful signal.

The study still has limits. Seventy-six patients and two comparison physicians make for a small sample. "Nominally better" is a cautious phrase that stops short of claiming large, statistically decisive margins 1. The comparison doctors were internists rather than emergency specialists 1. The results show capability, not readiness for unsupervised deployment.

The legal vacuum around AI-assisted diagnosis

The research has moved faster than the rules governing it. Guidance aimed at physicians stresses that FDA authorization, whether through 510(k), De Novo or premarket approval, is not a malpractice safe harbor. It also does not establish that a tool benefits patients in a given local setting and does not decide civil liability 3. A clinician who relies on an FDA-cleared tool is therefore not automatically shielded if that tool contributes to harm.

The same guidance separates several distinct legal questions that tend to get blurred together: professional negligence, institutional liability, product liability, contract terms with vendors, privacy and regulatory compliance 3. It warns against treating any one U.S. state's health-AI rules as a national standard. It also cautions against confusing enacted law with proposed policy, mock-juror research or academic scholarship 3. Put plainly, there is no single, settled framework that tells a doctor, a hospital or a vendor who answers for an AI-influenced diagnostic error.

Exposure from both directions

Legal scholarship cited in that guidance, by Mello and Guha (2024), describes physician exposure from two sides 3. One risk is using AI incorrectly, including over-relying on its output, the pattern known as automation bias 3. The Harvard results sharpen the other risk. If models reliably match or outperform clinicians on real cases, a future plaintiff could argue that ignoring an available AI suggestion fell below the standard of care.

That is the tension this study brings into focus. The better the models get, the more pressure there is to consult them. The more clinicians consult them, the greater the danger of deferring to a confident but wrong answer. The lupus example cuts both ways: an AI hypothesis can rescue a stalled workup, but a plausible-sounding wrong theory could just as easily derail one.

The takeaway

The Harvard work is strong evidence that reasoning models have become serious diagnostic aids, even on unfiltered clinical data. It does not settle how they should be used, documented or insured. Until courts, legislatures and insurers fill in the gaps, practical defenses will rest with clinicians and institutions. These include careful documentation of how AI output informed a decision, clear vendor contracts on indemnity, and explicit questions to insurers about coverage 3. The capability question is being answered quickly. The accountability question is not.