Mistral Large 4 Benchmarks: Big Leap, Still Behind China's Best

What Mistral announced

Mistral has introduced Mistral Large 4, a flagship model nicknamed "le Chonk." The French company describes it as the most capable open model built outside China and says it is among the best open models anywhere 4. The model has 1 trillion parameters, and Mistral highlights its strength in cyber, coding, manufacturing, finance and multimodal work 4. It is launching first as a preview for developers, "cybersecurity leaders" and state authorities, with a wider release planned later this month 4.

The timing matters. Open models, which anyone can modify and self-host, have become a front in the US–China technology contest, and the strongest of them have come from Chinese labs 4. A credible European entrant would be strategically useful. The open question is whether Large 4 is actually as strong as Mistral claims.

A real improvement

Large 4 is a large step up from its predecessor. On the Artificial Analysis (AA) Intelligence Index, Mistral Large 3 scored 9.3 last December, while Large 4 scores 38.4, roughly four times higher 1. That figure is well above the 26 median for models in its price tier 1. It also places Large 4 far ahead of US open models such as Nvidia's Nemotron 3 Ultra, which scores 23 3.

The price is aggressive. During the preview sale, Mistral charges $0.68 per million input tokens and $2.09 per million output tokens 1. That works out to about $1.13 per AA task at list price, or around $0.57 at the sale price 1.

Where the claims stop matching the data

The "best open model" framing weakens once independent scores come in. Several Chinese models sit above Large 4 on the AA index:

• Kimi K3: 43.6 1
• Qwen 3.8 Max: 45.4 1
• Others: Large 4 trails GLM-5.3, Kimi K3 and MiMo-V2.6-Pro by six to eight points 3

One reviewer summed up the result as a model that lands mid-pack 1.

The coding claim draws the sharpest criticism. Mistral says Large 4's 49.8% on its combined Coding Agent Index puts it ahead of DeepSeek V4 Pro 0813 and Qwen 3.8 Max 23. On Terminal-Bench 4.0, however, AA records Qwen 3.8 Max at 38.9%, while Mistral's own reported figure for Large 4 is 28.3% 2. That is a ten-point deficit on a published harness. The lead Mistral describes rests on a composite that cannot yet be checked independently 2. Other headline numbers are also self-reported, including 59.9% on AutomationBench, a set of 657 business workflows across tools like Gmail, Sheets, Slack and Salesforce, and 1,393 Elo on AA-Briefcase 3.

The sources do not agree on every number. One lists Qwen 3.8 Max at 45.4 1. Another table shows a Qwen 3.8 2.4T A95B variant at 40 with downloadable weights, and rounds Kimi K3 to 44 3. This affects the comparison, because "best open model" depends on which rivals count as open. Even against the lower Qwen figure, though, Large 4 still ranks below at least two Chinese open-weight models 3.

The weights problem

There is a more basic issue with the "open" label. Large 4's weights are not downloadable yet. They are promised for the end of October 2026 13. Kimi K3 and the Qwen variant above can be downloaded today 3. CNBC's account of a wider release later this month 4 appears to describe broader access to the model, not necessarily a weights release. Until the weights ship, Large 4 is open in name only.

Other trade-offs

The independent data also show some practical drawbacks [1]:

• Hallucinations: a 41.9% hallucination rate on AA-Omniscience.
• Verbosity: it used about 200 million tokens to complete the AA index, against an 81 million median. That reduces the real benefit of its low per-token price.
• Speed: it produces 116 tokens per second, but takes about 18.7 seconds to reach its first answer token.

The model's strongest areas, by Mistral's own charts, are cyber tasks without refusals, legal agent work and visual grounding 1.

The price comparison is mixed. Qwen 3.8 Max costs $2.00 per million input tokens and $6.00 per million output tokens, about three times Large 4's price, and it leads on every published measure 2. Whether that premium is worth paying depends on the workload.

Our read

Mistral is a serious competitor again, and among Western open models it may well be the leader. It is not the world's best open model. The specific coding claim conflicts with the one comparable public benchmark, and the "open" label depends on weights that are months away.

For European buyers concerned with data sovereignty, or teams focused on cost, Large 4 looks like a reasonable option. Buyers chasing top capability should treat Mistral's self-reported figures as unproven until independent evaluators can test the released weights.