GPT-6 Astra Benchmarks: Independent Tests Muddy the Lead

A launch, then a crowded scoreboard

OpenAI introduced GPT-6 Astra on September 3, 2026. The company pitched gains in coding, research, computer use and complex multi-step work, along with the ability to produce documents, spreadsheets and presentations that follow user templates and adjust as requirements change. 5 Within days, the model was being measured against Google DeepMind's Gemini 4 Argon, Anthropic's Claude Fable 5.1 and Claude Opus 5.5, and OpenAI's own GPT-6.1 Sol. 14

The results do not support a simple "best model" claim. Depending on who runs the test and what it measures, Astra is a narrow leader, a co-leader, or a runner-up. Its clearest advantage looks economic rather than raw capability.

Google's table favors Argon

Google's comparison covers 19 benchmark rows across knowledge work, agentic coding, ML engineering, science, long context, computer use, multimodal tasks and cybersecurity. Argon places first or ties for first on 14 of them. 4 The headline figure is DeepSWE v1.1, a long-horizon software engineering test, where Argon scores 77.9% against Opus 5.5 at 74.2%, Astra at 74.1% and Fable 5.1 at 67.4%. 14

In professional-work evaluations the gaps grow. Argon leads the Vals Index (68.9% vs Astra's 63.1%) and AutomationBench (51.3 vs 41.4). Its lead on Harvey's Legal Agent Benchmark is especially wide, at 19.6 against Astra's 5.4. 4

Astra still wins some rows in that same table. It leads FrontierSWE v2 (65.5% vs Argon's 55.0%), OSWorld-2.0 computer use (72.6% vs 69.2%), and Terminal-Bench 4.0 by a hair (58.2% vs 57.4%). It ties Argon on CWE-bench vulnerability remediation at 68%. 1 Astra also leads Terminal-Bench Science 0.1 at 68.1%, and OpenAI recommends it for the hardest research work. 1

These are vendor-reported numbers on a vendor-chosen benchmark set, and Argon is barely available to the public yet. 4 A first-or-tied finish on 14 of 19 rows is a strong showing. It is not a verdict users can easily check for themselves.

The independent picture, and a discrepancy

Artificial Analysis, which runs its own aggregate indices, gives Astra a better review than Google's table does. In its September 9 writeup, Astra at max effort scores 53 on the Intelligence Index, tying Claude Fable 5.1 and gaining 6 points over GPT-5.6 Sol. 2 It also ties Fable 5.1 on the firm's Coding Agent Index. That result is credited to the lowest token use of any agent measured. 2

MarkTechPost's roundup reads the independent data differently. It cites Astra at 61 on the Intelligence Index with Fable 5.1 five points higher, and a Coding Agent Index edge for Fable 5.1 in Claude Code, 70 to 67. 1 Those figures conflict with Artificial Analysis's own published tie. The cause could be differing index versions, effort settings or timing, but the sources do not settle it. Readers should treat the primary Artificial Analysis post as the more authoritative account of its own results.

Cost is Astra's real story

On price, the sources line up. Artificial Analysis says every Astra reasoning-effort level sits on its intelligence-versus-cost frontier. It found Astra matches Fable 5.1's Intelligence Index score at about 40% of the cost per task ($3.26 vs $7.63) and its coding score at about 60% of the cost. 2 Astra is not cheap, though. At max effort it runs roughly 60% more per task than GPT-5.6 Sol, partly because of a higher per-token price. 2

For buyers, that may matter more than a point or two on any single leaderboard. A model that ties the top competitor at well under half the price changes procurement math even if it is not first on every benchmark.

Beware the leak cycle

The noise around these launches is part of the story. A Reddit post circulating supposedly leaked "Gemini 4 Pro" figures listed Astra at 86.9% on DeepSWE and Argon-class scores near 89%. 3 Commenters quickly pushed back, noting that Astra scores around 74% and that no model reaches 87%. 3 That matches the published tables. 14

The reading

GPT-6 Astra is a genuine frontier model with specific strengths in frontier-level software engineering, computer use and research-heavy terminal work, plus a notable cost advantage. 12 It is not a clear overall leader. Google's own numbers put Argon ahead on most professional and long-horizon tasks, and independent indices at best show Astra tied with Claude Fable 5.1. 24 The right choice depends on the job, and on whether the benchmark comes from the vendor or from an independent evaluator.