GPT-6 Astra Benchmarks: #1 Rank, but Users Prefer Claude Fable

A leaderboard win with an asterisk

OpenAI's GPT-6 Astra has reached the top of a major public leaderboard, but its lead is less clear than the ranking suggests. In Arena's Code Arena: WebDev table, Astra (Max) sits at #1 with 1800 points, ahead of Anthropic's Claude Fable 5.1 (Max) at 1758. 1 Arena's own breakdown of the matchup between the two models complicates that result.

In anonymous head-to-head battles judged by Arena users, Fable 5.1 wins 43.5% of the time, Astra wins 29.0%, and 27.4% end in ties. 1 Arena stated it directly: despite Astra's #1 spot, Fable 5.1 is the model people prefer when the two are compared side by side. 1 Against the rest of the field, Astra wins most battles, including 72.5% against OpenAI's own GPT-5.6 Sol (xHigh). 1

The split appears on other boards as well. Arena's September 11 WebDev snapshot ranked Astra first, Fable 5.1 second and Claude Opus 5 Max third. Its separate Agent Arena leaderboard, dated September 9, put Fable 5.1 Max first and Astra second. 2

Manipulation, or how ratings work?

Some users read the gap as foul play. One reply to Arena's post called it "peak benchmark manipulation." 1 A more mundane explanation is more likely. Arena's rankings combine results across many opponents. A model that beats a broad field by wide margins can earn a higher overall score than a rival that beats it directly but does less well against everyone else. Astra's lopsided wins over weaker models would push its rating up even while it loses the one matchup that matters most to buyers choosing between the top two.

That explanation is an inference from how pairwise rating systems usually behave, not a confirmed account of Arena's methodology. The practical point holds either way. The single number at the top of the table and the head-to-head preference answer different questions. If you are choosing between Astra and Fable 5.1 for web development, the direct comparison is probably the more useful signal.

Arena seems aware of this. Its co-founders, Anastasios Angelopoulos, Wei-Lin Chiang and Ion Stoica, are building the company around splitting "best" into specific tasks, costs and confidence intervals, and Arena is becoming part of how developers decide which models to buy. 2 Publishing a win-rate chart that undercuts its own #1 ranking fits that approach.

The cost side of the ledger

Price adds another variable. Astra launched on September 3, 2026, at $10 per million input tokens and $50 per million output tokens. It has a context window of roughly 1.05 million tokens and five reasoning-effort levels, from low to max. 4 That is about 2.5 times the price of GPT-5.6 Sol, which costs $4 and $20. 3

Per-token price does not tell the whole story. Artificial Analysis found Astra uses about 10% fewer output tokens than Sol on general intelligence benchmarks, which does not offset the higher price. 3 On coding-agent evaluations, though, Astra used about a third as many tokens as Sol. At max effort, that brings its cost per coding task to roughly Sol's level while scoring a couple of points higher. It also comes in at less than half the cost of Claude Opus 5 for similar results. 3 So Astra may look expensive on a rate card while being competitive on cost per task for coding work, which is the category where its human-preference standing is weakest against Fable.

Independent trackers point the same way

Other measurements also show a contested field rather than a clear leader. Astra scores 52.67 at max effort on Artificial Analysis's Intelligence Index. 4 That tracker has Astra tying Fable 5.1 on both its Intelligence and Coding Agent indices, not beating it. 5 Google's own comparison shows its new Gemini 4 Argon ahead of Astra on 14 of 19 published benchmarks, including DeepSWE v1.1 (77.9% vs. 74.1%) and the Vals Index. 5 Vendor-published comparisons deserve skepticism no matter who publishes them. Still, the pattern matches Arena's: Astra's clearest advantages are in computer-use tasks such as OSWorld-2.0 and in terminal-based science work, not across the board. 5

The reading

Astra is a frontier model with real strengths and an unusual cost profile that favors coding agents. It is not a clear overall winner. Its WebDev #1 ranking is accurate as an aggregate score, but it hides a head-to-head loss to Fable 5.1 among the people actually voting. For teams choosing a model, the takeaway goes beyond Astra versus Fable. Top leaderboard positions now come down to small margins on specific tasks, and the decision depends on which tasks you run, what you pay per completed task, and which pairwise comparisons matter to you.