Frontier AI Earnings Prediction Beats Analyst Consensus in New Test

What happened

AI startup Samaya says frontier language models have, for the first time, predicted corporate earnings more accurately than professional analysts. The models ran inside Samaya's own finance-specific harness, which feeds them financial data restricted to what was available at each point in time.1 The test covered 456 second-quarter earnings releases. Model forecasts were compared with two benchmarks: the raw analyst consensus, and a tougher version of that consensus adjusted for analysts' known bias.1 Several models beat the raw consensus. Only the newest models (GPT-6 Astra, Claude Fable 5.1 and Claude Opus 5.5) also beat the bias-adjusted version, which Samaya calls an inflection point.1

The headline is easy to share, but the design of the evaluation matters more than the result. Read alongside the other evidence on AI forecasting, the study tells us as much about how hard it is to evaluate frontier models fairly as it does about whether AI can now out-analyze Wall Street.

How the test was built

Each model made its forecast five trading sessions before a company reported. It predicted four figures: revenue, gross margin, operating income and adjusted earnings per share.1 The study covered companies that reported on or after July 14. That date falls after the training-data cutoff of every model tested, which addresses the obvious worry that a model had simply memorized the answers.1 To qualify, a company needed a market value above $5 billion and estimates from more than eight brokers, and the sample covered every sector.1

Samaya added two further safeguards that will look familiar to anyone who works on model evaluations. The first is a "point-in-time gate" built into the harness. It blocks any document published after the forecast date, and Samaya says it is enforced with an authentication token so the model cannot get around it.1 The second is a test with the analyst consensus removed entirely. Samaya stripped out structured estimate data and deleted any sentence in retrieved documents that revealed a consensus figure.1

Most AI headlines skip this kind of work. It treats the model as a possible adversary of its own evaluation, an agent that might find and use leaked future information if allowed. Red teams take the same stance when they look for reward hacking, and seeing it in a commercial finance study is a good sign.

What the results show

GPT-6 Astra led on the most measures, including revenue error, overall error and hit rate. Fable 5.1 and Opus 5.5 did best on how well their predicted surprises tracked the actual ones.1 The pattern Samaya emphasizes is that error falls steadily as models become more capable.1

The breakdown tests are the most useful part of the study. With no outside data, even the strongest models did poorly. Fable 5.1 could not make use of public information it had absorbed during training.1 Data from the start of the quarter raised performance by 25 percentage points. The latest data added another 12 to 16 points, which Samaya says is about the gap between mid-tier and frontier models.1 Expert-written instructions led models to research 1.6 to 2.7 times more and reduced their errors.1 With the consensus hidden, weaker models fell off sharply. GPT-6 Astra scored almost the same, possibly because it rebuilt the consensus from public sources.1

The models did best on revenue and gross margin. They were weaker on operating income and adjusted EPS, which depend on the other figures and on company-specific adjustments.1 Samaya's review of the models' reasoning found that their wins usually came from finding new evidence and being willing to depart from the consensus. Samaya presents this as anecdotal.1

Where the claim is weaker than it sounds

The human baseline is easy to beat. Samaya itself found the raw consensus a weak benchmark. Analysts tend to lower their estimates before earnings, so companies beat the consensus more often than they miss it.1 Its tougher baseline adds each company's historical median surprise to the consensus, but only when that surprise is positive. The adjustment can raise the consensus but never lower it.1 That is a reasonable fix, but it is not new. Academic work has long shown that machine learning can correct analyst bias. One study found a neural network improved consensus accuracy by 24% over a linear model and called the direction of the surprise correctly 68% of the time. An observer on Hacker News argued that analysts' directional calls are often no better than a coin toss.7 Beating the consensus is a real result, but it is not the same as beating the best human forecasters.

The test covered one unusual quarter. All 456 releases came from a single reporting season, and that season was far from normal. Bloomberg Intelligence found that 86% of S&P 500 companies beat analyst expectations, the highest share since 2021.19 LSEG reported the same 86% figure across 492 companies, against a long-run average of 67.5%.13 In this analysis's reading, a baseline built from each company's past surprises would have underestimated a quarter with this many beats. Any model inclined to forecast above the consensus would have been rewarded. That does not erase the results on correlation or on below-consensus calls, where the best models did especially well.1 But one season driven by AI spending is not enough to say AI has passed humans for good. More quarters, including ones where companies miss, are needed.

Results varied a lot from run to run. When GPT-6 Astra and Fable 5.1 each ran five times on 100 companies, the best run was much more accurate than the average run. Averaging the runs did not help much.1 An investor using one of these systems would need to know which run to trust, and nobody can tell that in advance.

The company running the test sells the product. Samaya built the harness, chose the baseline and wrote the results, and it concludes that frontier models need "the right harness" to succeed.1 Independent replication should be expected before the claim is accepted.

How it fits the wider forecasting picture

The broader evidence points the same way, even if this particular "first" is open to debate. The Economist reported in September that AI systems now beat some of the best human forecasters.9 In July, the Forecasting Research Institute said its top models were "statistically indistinguishable" from elite human forecasters.3 AI bots took first and second place in a recent Metaculus tournament, which had never happened before.3 A Metaculus analysis also found that custom prompting, searching the web and combining several forecasts could be worth about nine months of progress in base models.3 That echoes Samaya's finding that the harness and the data matter as much as the model.1

The same shift is showing up in other professional work. Mercor reported that Claude Opus 5 completed four month-end accounting tasks without error on every attempt. Licensed CPAs averaged about 37%, and the model was dozens of times cheaper per task.4 Stanford's 2026 AI Index notes that frontier models now match or beat humans on PhD-level science and competition mathematics, though their abilities remain uneven.8

Not everyone agrees on how deep this goes. IBM's reporting cautions that forecasting bots do best on clear, well-defined questions and are harder to judge on messy business problems. Many practitioners prefer pairing humans with AI over full automation.3 On Hacker News, one commenter said forecasting simulations show little difference in skill between models, because results depend on how other agents behave. Others noted that forecasts are highly sensitive to modeling choices.7

Why safety researchers should care

For the alignment and evaluation community, there are three lessons.

First, capability depends on the harness. Samaya's results reflect the model plus its tools, data access and expert instructions.1 Tests that measure models without those tools will understate what they can do once deployed. The same issue arose when Anthropic said it was holding back Claude Mythos Preview because the model, given the right setup, could find thousands of zero-day software vulnerabilities.6

Second, contamination controls are becoming standard. A dated knowledge cutoff, a retrieval gate enforced by code and a test with the consensus removed together form a template that evaluations of forecasting and agent capabilities should copy.1

Third, the weak point is the human baseline. Whether AI "beats experts" depends on which experts it is compared with and how their numbers are adjusted. This study's baseline was more careful than most, but it came from one quarter, in one market regime, and was designed by the company with the most to gain.

This analysis reads the study as a well-built signal, not a settled milestone. AI earnings prediction is probably near or slightly past the analyst consensus, and that line will hold up better once the result is replicated in a quarter when companies miss.