Mistral Large 4 Benchmarks Trail Its 'Leapfrog' Ambitions

A trillion parameters, and a bold pitch

On October 6, Mistral AI opened a public preview of Mistral Large 4, a natively multimodal mixture-of-experts model. The French lab has nicknamed it "Le Chonk." 34 Mistral framed the release as an attempt to leapfrog both American and Chinese rivals. It also tied the launch to what French president Emmanuel Macron has called "a third way in AI," an alternative to closed models that can be switched off and to open models that mostly come from China. 5

The headline numbers are large. The model has roughly one trillion total parameters and is a hybrid instruct-and-reasoning system. 4 It accepts text and images through a 1.6-billion-parameter vision encoder and returns text. 1 It was trained across more than 160 languages, including every official EU language. 14 The weights are promised by the end of October, reportedly on October 27. That would make it the largest open-weight model a European lab has released. 34

The independent picture so far is more modest than the launch rhetoric.

What third parties are measuring

Artificial Analysis gives the preview a 38 on its Intelligence Index. That puts it behind two Chinese open-weight flagships, GLM-5.3 at 45 and Kimi K3 at 44. 1 LMArena has not yet listed the model. 1 On the BenchLM leaderboard, Large 4 ranks 71st out of 889 models, though that ranking draws on a limited set of benchmarks. 3

The task-level results are uneven rather than weak. Large 4 leads open-weight models on the Harvey Legal Agent Benchmark with 15.83%. It ties DeepSeek V4 Pro at 67% on FinWorkBench and scores 93% on Cybench, a strong cybersecurity result. 3 On Dense200 image recognition it reportedly edges out GPT-6 Astra by about a percentage point. 3

Coding is the clear gap. Large 4 scores 61.7% on DeepSWE v1.1, while GPT-6 Astra and Gemini sit around 74%. Its Terminal-Bench 4 scores are in the low-to-mid 20s. 3 Agentic coding is where much enterprise demand is concentrated, so this is a meaningful shortfall.

Inconsistent spec sheets

Even basic specifications vary depending on where you look. The announcement says "1 trillion" parameters with 49 billion active per token. Mistral's model card lists 1.05 trillion total and 52 billion active. 14 Mistral's documentation gives a 1-million-token context window. Artificial Analysis lists 524k for the preview API. 14 Reported training compute ranges from about 3,800 to 4,000 Nvidia Grace Blackwell GPUs in Mistral's European datacenters, over roughly two months. 14

Mistral cofounder Stock presented that compute figure as a strength. He described 4,000 GPUs as two to three times fewer than Chinese competitors use, and far fewer than closed-model labs. 5 Each discrepancy is small. Together, they suggest a preview that shipped before the documentation was finalized.

Pricing and the missing license

The list price is $1.36 per million input tokens, $0.14 for cached input and $4.18 for output. During the preview, Mistral is charging roughly half those rates. 1

The bigger open question is the license. As of the preview, Mistral has not said what terms will govern the open weights. 123 The lab says its post-training method and full benchmark suite will be published alongside the weights. 2

Mistral's recent history shows why the license matters. Mistral Small 4 uses Apache 2.0 with no revenue limits. Mistral Medium 3.5 uses a modified MIT license that bars companies with more than $20 million in monthly revenue from using it without a separate commercial agreement. 2 If Large 4 follows the Medium approach, the large enterprises most likely to need a trillion-parameter self-hosted model would also be the ones that have to negotiate.

The reading

The "leapfrog" claim does not hold up on current independent evidence. On aggregate intelligence scores, Large 4 trails the best Chinese open-weight models by a clear margin. On coding, it trails the top closed models by more. 13 Calling the model a disappointment would also miss the point. Its results in legal, finance, security and vision work are competitive, and its broad language coverage serves a European market that Chinese labs and US labs often treat as secondary. 13

In practice, Large 4 is a model chosen for its weights, its origin and its specialist strengths, not for raw leaderboard rank. It could be an attractive sovereign option for regulated European industries. Whether that holds depends on a license Mistral has not yet published. Until the weights and terms arrive, the most important facts about the model remain unknown.