Mistral released the weights of Mistral Large 4 on 29 October and says it is now the best open-weight model for coding. The first independent numbers, from benchmarking firm Artificial Analysis, are more modest. They put it second on a coding agent ranking and score it lower than Mistral does on two tests Mistral leads with.
What Mistral released
Large 4 is a mixture-of-experts model: each token passes through a few specialist sub-networks instead of the whole model. The Mistral Large 4 model card lists about 1 trillion total parameters, with 49 billion active per token. It also lists a 1 million token context window.
The weights ship under a modified MIT license. In its open-weight release post, Mistral says most uses are covered, but a business running a model-as-a-service offering needs a separate commercial license. API prices are $0.70 per million input tokens and $2.10 per million output tokens.
What Mistral claims
In the release post, Mistral reports 19.7% on Cognition’s private FrontierCode 2.0, which tests whether a maintainer would merge an agent’s code change. It reports 41% on Terminal-Bench 4.0, a test of agent work in a command-line shell, up 13 points from the preview. It also reports 48% on the Artificial Analysis Coding Agent Index, “effectively tied” with GLM-5.3 while using 47% fewer tokens per task.
Mistral also flags a risk of its own. After monitoring preview traffic, it says it sees “a potential for uplift to conduct real attacks” and is changing its guardrails.
What the independent runs show
In its Coding Agent Index results, Artificial Analysis ranks Large 4, running in Mistral’s Vibe coding agent, second among open-weight entries at 48. GLM-5.3 in Opencode scores 49 and DeepSeek V4.1 Flash 1017 scores 46. The token claim holds: 8.5 million tokens per task, against 15.9 million and 16.7 million.
Single benchmarks diverge more. In its Intelligence Index evaluation, Artificial Analysis measured 25% on Terminal-Bench 4.0 and 15% on FrontierCode 2.0. Mistral labels its 41% chart as part of the Coding Agent Index and does not say which agent setup produced its 19.7%. We could not confirm how much of each gap comes from setup rather than the model.
Setup matters because these scores measure a model plus its agent software. The Artificial Analysis coding agent page says model choice, settings and execution configuration “can materially change outcomes”.
Some independent numbers favor Mistral. Artificial Analysis measured a 28% hallucination rate, down from 41% for the preview, and cost per Intelligence Index task fell 63% to $2.06. Its overall Intelligence Index score is 26, per the Artificial Analysis model page. The firm’s evaluation puts that 6 points behind the leading open models.
What it takes to run
Open weights do not mean small hardware. The vLLM serving recipe puts the FP8 checkpoint at about 1.06 TB and validates it on setups such as 8 Nvidia B300 GPUs. It tested context only up to 256k tokens.
The Hugging Face launch post reports a 2-bit version on one Apple M3 Ultra with 512 GB of memory, generating about 15 tokens per second.
What to do this week
If you are weighing Large 4 for coding agents, run it in your own agent software on your own repositories. Log tokens per task next to pass rate. Treat the 41% and 19.7% figures as Mistral’s own until a third party reproduces them, and read the model-as-a-service clause before selling hosted access.
Sources
Get the weekly email
The week's most useful AI news, a tutorial and an interview.
Lars Strother
Lars Strother is the senior editor for models and hardware at AI Observer, covering models and research, chips and hardware, and robotics. He came to journalism from engineering and still reads the benchmark tables before the press release. His coverage is measured and skeptical, and he wants to know what was tested, on what, and against which baseline.
Spotted a mistake? Tell us on the corrections page.