By Lars Strother

. 5 min read

Four public leaderboards dominate talk about which AI model is “best”: Arena (formerly LMArena and Chatbot Arena), Artificial Analysis, LiveBench and Scale’s SEAL leaderboards. They regularly put different models on top because each one measures a different thing. Before you pick a model from any of them, check what was measured, who graded it and at what settings.

On 3 November 2026, the boards did not agree on first place. Artificial Analysis put Gemini 4 Carbon (high) first with an index score of 59. LiveBench put Claude Fable 5.1 first at 85.7, a tenth of a point ahead of GPT-6 Astra. All three models are near the top everywhere, but the order changes depending on which board you read.

Four boards, four different questions

Arena: which answer do people prefer?

Arena shows two anonymous models the same prompt and asks the visitor to pick the better answer. Per its FAQ, only votes cast while the names are hidden count. The votes feed a Bradley-Terry model, a statistical method for turning many head-to-head results into one rating, similar to chess Elo.

Its leaderboard policy says a model needs at least 1,000 votes before it is listed, and that providers may test several unreleased variants under code names. Since 16 May 2025, the text and vision boards default to “style control”. That adjustment, explained by Arena, removes the effect of answer length and markdown formatting, because voters tend to favor longer, well-formatted replies.

Arena also shows a rank spread next to each rank: the best and worst position a model could hold given the confidence intervals. Two models whose spreads overlap are effectively tied. Arena now runs separate boards for text, vision, search, coding, image, video and agents, listed in its public dataset. The agent board uses a different scoring method from the text board.

Artificial Analysis: one index from nine tests

Artificial Analysis runs the tests itself. Its Intelligence Index v5.0 combines nine evaluations in four categories (knowledge work, coding, science and general), each weighted at 25%. Several of those tests use private question sets, and several are graded by another language model acting as judge.

Its methodology section also covers price and the speed users actually get from each API provider. That makes it the only board of the four that puts quality, cost per task and tokens per second side by side. Its own caveat is worth keeping: the index is mostly English text, so it says little about other languages.

LiveBench: graded work, run once

LiveBench is run by Abacus AI, according to its home page. The 2.0 version has seven suites, according to its methodology page. Five are agentic, meaning the model uses tools to finish an end-to-end task such as a coding, design or web research job. The other two are short question sets with one correct answer.

The overall score is a weighted mean: long agentic work counts twice and coding and design 1.5 times. Releases up to 14 October 2026 weighted every suite equally, so older overall scores are not comparable with the current ones. Each cell comes from a single run at the model’s highest effort setting, and a task that errors out twice scores 0.

LiveBench calls itself “contamination-free”, meaning the questions did not leak into training data. We could not confirm that independently, and the scores still depend on its own test setup.

Scale’s SEAL leaderboards: private tests, one shot per lab

Scale launched the SEAL leaderboards on 29 May 2024 with private datasets built by vetted domain experts. Its leaderboard page now lists more than 20 benchmarks and over 100 models.

The key rule: a model is only eligible the first time its developer sees the prompts. Later models from the same lab can be added in exceptional cases but are grayed out with an overfitting warning, per the launch post. The upside is that labs cannot tune against the test. The downside is that outsiders cannot inspect the questions or rerun the scores.

Why the rankings disagree

The first reason is the task. A vote on a chat answer, a long agentic task and a private expert exam reward different skills. In Arena’s agent board published on 2 November, Claude Opus 5.5 (Max) ranked first and Claude Fable 5.1 (Max) fifth, while LiveBench put Fable first.

The second is the setting. Reasoning effort controls how long a model may “think” before answering, and it moves scores a lot. On Artificial Analysis, Claude Opus 5.5 scores 58 at “max” effort and about 35 at “low”. A board that tests one setting and a board that tests another are comparing different products under one name.

The third is aggregation. LiveBench and Artificial Analysis each choose weights for their suites, and LiveBench changed its weights within one month. Arena reports confidence intervals, and Scale prints error bars such as GPT-6 Astra’s 60.60±3.00 on Humanity’s Last Exam (Diamond). A one-point lead inside those margins is not a real lead.

The last is who grades. Arena relies on its visitors, who are not a random sample of your users. Artificial Analysis and LiveBench use automated checks and, in places, language model judges. Scale uses human experts plus model-assisted grading.

Grading and setup also explain gaps between labs and outside testers. When we covered Mistral’s coding claims for Large 4, Artificial Analysis measured lower single-test scores than Mistral reported.

Which board to check for your use case

  • A chat assistant or writing tool for the public: start with Arena’s text board with style control on, and look at the rank spread, not only the rank.
  • An API product where price and speed matter: use Artificial Analysis, and compare the model at the effort setting you will actually pay for.
  • Coding agents and long multi-step tasks: read LiveBench’s agentic and coding columns, plus Arena’s agent and coding boards.
  • A narrow professional domain such as legal or finance reasoning: check whether Scale has a SEAL board for it, and note grayed-out entries.

None of these replaces a test on your own data. When a lab announces a model with only its own benchmark table, treat it as a claim until one of these boards runs it. That was the case when Meituan published LongCat-2.5 scores with no independent results yet.

What to do this week

Pick the two boards that match your use case from the list above and shortlist the three models that sit near the top of both. Then run 20 to 50 real prompts from your own work through each, at the effort setting you plan to pay for, and score them blind. That small test will tell you more about your workload than a one-point gap on any public leaderboard.

Sources

Lars Strother

Lars Strother is the senior editor for models and hardware at AI Observer, covering models and research, chips and hardware, and robotics. He came to journalism from engineering and still reads the benchmark tables before the press release. His coverage is measured and skeptical, and he wants to know what was tested, on what, and against which baseline.

Spotted a mistake? Tell us on the corrections page.

Conversation

Every comment is reviewed before it appears, so there can be a short delay.