Mistral Large 4 Benchmarks Test the 'Le Chonk' Open-Weight Claim

What Mistral Launched

Mistral AI has put its new flagship, Mistral Large 4, into public preview. The French lab has nicknamed it "Le Chonk" and calls it its largest and most capable model to date 4. The model is a natively multimodal mixture-of-experts system with roughly a trillion total parameters. Only about 49 billion are active for any given token 34. One detailed breakdown puts the total at 1.05 trillion and adds a 1.6-billion-parameter vision encoder 1.

The context window grows to 1 million tokens, up from 256K on Mistral Large 3 1. The API is live now under the model ID mistral-large-4 1. Open weights are slated for late October, with October 27 cited as the target date after a staged testing period 14. The license is listed as "Open," but the precise terms have not yet been spelled out 4.

Mistral says it trained the model on Nvidia Grace Blackwell GPUs in its own European datacenters. Its blog cites 3,800 chips, while other reports round that to 4,000 3.

The Headline Claim

Mistral's pitch is pointed. It presents Le Chonk as by far the most capable open-weight model built outside China, and says it comes "very, very close" to some proprietary systems 5. The company benchmarks directly against Chinese open-weight leaders, including DeepSeek V4 Pro, Qwen3.8 Max, Kimi K3 and GLM-5.3. It claims to be "significantly outperforming" any open-weight model from the US or Europe 3.

The model is general-purpose but tuned for coding and cyberdefense. It also targets niche domains such as manufacturing, finance and electrical engineering 5. Mistral leans on cybersecurity as a differentiator and reports a 93% score on a cyber benchmark in its own materials 3.

There is also a political dimension. US officials have accused Chinese labs of leaning on distillation, which means training models on the outputs of larger rivals, to catch up with OpenAI and Anthropic. Mistral stresses that it trained Le Chonk from scratch 5. That framing matters as access to frontier AI becomes a geopolitical issue. Mistral clearly wants to be seen as proof that Europe is "still in the race" 5.

What Independent Numbers Show

Outside measurements are more sobering. VentureBeat ran an independent cross-check and found Le Chonk at 62% on the DeepSWE coding benchmark, compared with 69% reported for GLM-5.3. That result suggests Chinese open-weight models still hold a coding edge at launch 3.

On the Artificial Analysis Intelligence Index, the preview scored 38. That placed it last among the 25 models tracked 2. It sits well behind the top closed models, though it remains notable among non-Chinese open-weight releases 2.

These results do not directly contradict Mistral's narrower claim. Being the best open-weight model outside China is a different bar from beating China's best or matching closed frontier systems. But they make the "very, very close" to proprietary models framing harder to accept at face value.

Pricing Confusion

Price is where Mistral has the clearest case, but even the figures are inconsistent. One source lists preview pricing of $0.68 per million input tokens, $0.07 for cached input, and $2.09 per million output tokens 1. Another cites $1.36 per million input and $4.18 per million output 2.

The second set is exactly double the first. That could reflect a preview discount versus standard rates, or a change between reports, but none of the available reporting settles it. Either way, both sets place Le Chonk in the inexpensive tier relative to closed frontier models 2.

The sparse architecture helps explain the low cost. Activating 49 billion parameters per token keeps serving costs closer to a mid-size dense model than the trillion-parameter headline suggests 3.

Our Read

Le Chonk looks like a real step forward for Western open-weight AI, and a credible one on cost and context length. It is not a leapfrog.

Mistral's marketing is carefully worded. "Best outside China" is a claim that concedes the frontier of open models currently sits in Beijing and Hangzhou. Early independent data supports that concession more than it supports any implication of parity with closed leaders.

The real test comes when the weights ship. Developers will then be able to run their own evaluations, and the license terms should become clear.

For enterprises that care about data sovereignty, provenance, and a model trained without distillation controversy, Le Chonk may be compelling regardless of leaderboard rank. For anyone who simply wants the strongest open coding model, the numbers so far point elsewhere.