AI Brain Aneurysm Detection Rises 39% in Northwell Shadow Study

What the study found

Researchers at Northwell Health ran an FDA-cleared AI algorithm silently in the background on 3,856 brain CT angiography exams. The software flagged 55 confirmed intracranial aneurysms that were not in radiologists' reports. Measured against the 140 aneurysms that radiologists correctly reported, that is a 39% relative increase in detection.13 The Journal of the American College of Radiology published the work on September 15, 2026.18 The algorithm comes from the Tel Aviv–based company Aidoc.11

The 39% figure is a real result, but it is not the main thing hospitals should take from the paper. The more useful lesson is where the tool earned its keep and where it did not. Hospitals deciding whether to buy, switch on and pay for radiology AI need exactly that kind of operational evidence, and very little of it exists.

Sensitivity versus precision

On sensitivity, the AI beat the radiologists. It caught 84.6% of aneurysms that were actually present, compared with 71.8% for radiologists reading alone.11 On precision, the radiologists won. When they called an aneurysm, they were right 92.7% of the time. The algorithm's positive calls were right 78.2% of the time.12 The two had nearly identical specificity, about 98.5% for radiologists and 98.7% for the AI, so both were about equally good at clearing normal scans.13

The two also missed different things. Radiologists caught 30 aneurysms the software missed, which gives an AI-to-radiologist incremental detection ratio of 1.83.13 The AI produced 101 findings that radiologists had not reported. Expert review confirmed 55 of them and rejected 46 as false alarms.16 The aneurysms were different in size, too. More than half of the AI-only finds (31 of 55) were smaller than 3 millimeters, while the radiologist-only finds were mostly 3 to 5 millimeters.13

Lead author Shlomit Goldberg-Stein argued that small aneurysms can still matter, because rupture risk also depends on shape, location and patient factors.12 AuntMinnie reported that brain aneurysms affect up to 4% of the population and that about half of ruptures are fatal.18

Hospital setting made the biggest difference

The headline number hides a large difference between care settings. Inpatient scans were the best case. The AI added 18 aneurysms and generated seven false alerts.11 The study's metrics for inpatients were a relative enhanced detection rate of 78.3%, a "gain-to-pain" ratio of 2.57, and a number-needed-to-examine of 28.9.13

The emergency department came out roughly even, with a gain-to-pain ratio of 1.0 and a 37.1% detection gain. Outpatient imaging was unfavorable. The detection gain fell to 14.1%, the gain-to-pain ratio dropped to 0.67, and about 130 exams were needed to find each additional aneurysm.13 In practical terms, the AI added only four outpatient detections and raised more false alarms than true ones.12 Across all settings, the gain-to-pain ratio was 1.20.18

Matthew Barish, a Northwell co-author, said sicker inpatient and emergency populations produce more complex exams, which gives a second reader more chances to help.11 That explanation is reasonable but it was not directly tested. The practical consequence is clearer than the cause. A hospital that runs this tool the same way across all departments may get real value on the wards and mostly extra work in the outpatient clinic.

How outlets framed it

Coverage agreed on the core numbers but framed them differently. AuntMinnie led with the 39% gain and quoted the authors' conclusion that the results support clinical deployment.18 Radiology Business and a trade digest led instead with the uneven performance across settings. One went as far as saying the gain came almost entirely from emergency and inpatient cases.1920 Medical News Today, and the South African outlets that republished its story, put the most weight on what the study cannot show: whether finding more aneurysms actually leads to fewer ruptures or better outcomes.1213 A Gadget column added that the results come from one health system and one commercial algorithm and should not be assumed to apply elsewhere.16

The cautious framing is the right one. The study measured detection, not outcomes. A tool that finds many sub-3-millimeter aneurysms will lead to more surveillance imaging, more specialist referrals and more patient anxiety. For some of those patients, catching the aneurysm will change what happens to them. For others it may not. The study does not tell us which group is bigger.

There are also small inconsistencies in the coverage. The abstract's size breakdown counts 33 radiologist-only aneurysms, while the main results count 30.13 One newswire page labels the work a meta-analysis, though it was a single prospective study.15 Neither error changes the main findings.

Why the shadow-mode design matters

The authors called the study design "shadow mode." The algorithm processed every exam, but radiologists never saw its output, so it could not affect care. When the AI and the radiologist disagreed, independent neuroradiologists decided which was right.11 Natural language processing was used to pull findings out of the radiologists' reports.13 About 5% of the scans, 195 exams, contained an aneurysm.19

This design is valuable because it measures what the software would add in a real hospital, with ordinary patients, scanners and workflows, rather than on a hand-picked test set.16 It also has limits. The study adjudicated only the cases where AI and radiologist disagreed. The authors acknowledge that this "hybrid reference standard" shapes their estimates.13 If both missed the same aneurysm, nobody found it. Sensitivity estimates for both readers are therefore probably somewhat optimistic.

The second limit is behavioral. The study projects how radiologists and AI would perform together, but it never watched radiologists actually using the alerts. In real use, readers can over-trust a flag, wave off a correct one, or get tired of false alarms. The stated benefit of combining the two is an estimate, not a measurement.

The third limit is age. The exams were read between November 7 and December 19, 2023, almost three years before publication.18 Both the algorithm and radiology practice have likely changed since.

Regulation: clearance doesn't show real-world value

Elizabeth Rula, executive director of the Harvey L. Neiman Health Policy Institute, which is affiliated with the American College of Radiology, said health systems should judge AI by how it performs in real practice, not by its results in original testing. She also called for ongoing monitoring after deployment.1115 That argument goes to the heart of how these products reach the market.

As of September 2026, the FDA says it has authorized more than 1,600 AI-enabled medical devices.1 Its most recent quarterly update put the total at 1,614. Of those, 1,230, or 76%, are radiology devices, and the agency authorized 89 AI devices in the second quarter of 2026 alone.10 Nearly all AI devices are cleared through the 510(k) pathway, which requires showing substantial equivalence to an existing device rather than running new clinical trials.7 Independent analysts have pointed to a persistent gap between how fast clearances are growing and how much clinical evidence supports them.2 The FDA itself describes its list as not comprehensive.6

Aidoc sits near the top of that list among AI-only vendors, with about 31 clearances.3 One industry roundup says its software is in roughly 2,000 hospitals. In January 2026 it received what was described as the first FDA clearance for a foundation-model triage device, which covers 14 conditions on body CT.9 In June 2026, the FDA also gave Aidoc breakthrough designation for a chest X-ray tool that drafts reports.5 As the company's footprint grows, studies like Northwell's become more important, because clearance shows a tool works on a validation set, not what it will add in a given hospital's patient mix.

The Northwell paper does not make the case for loosening oversight, and it does not make the case against using these tools. It makes the case for treating FDA clearance as the starting point. Health systems should run local shadow-mode trials before switching a tool on, set it up differently for each care setting, and keep tracking false-alarm burden after go-live. The study's own metrics, such as gain-to-pain ratio and number-needed-to-examine, are a reasonable model for that tracking.18

Bottom line

The study shows that a commercial aneurysm-detection AI, working as a second reader on brain CT angiography, would likely find many lesions that radiologists miss, and that radiologists in turn catch lesions the AI misses.14 It also shows that the benefit depends heavily on the care setting, and that outpatient use did not clearly justify the extra alerts.20 The 39% figure will get the attention. For hospitals and regulators, the more lasting finding is that real-world value has to be measured site by site and setting by setting, and it is still unknown whether finding more aneurysms leads to better patient outcomes.12