How benchmark scores work
The directory shows two numbers Priors calculates itself. Neither is a model’s result: they describe the benchmark. Leaderboard scores stay exactly as published.
Quality
A score from 0 to 100 for how useful a benchmark is for telling current models apart. It’s a weighted sum of five parts, each also 0 to 100:
| Part | Weight | What it measures |
|---|---|---|
| Headroom and spread | 30% | Can it still tell top models apart? 75% is the room left above the top quarter of models, 25% is the gap between best and worst. A board with under 5 points between them caps at 35. Scores with no fixed scale, such as Elo, get a neutral 50. |
| Recency | 20% | Full credit for 90 days after the benchmark's release, then falling to zero at two years. Neutral 50 if we haven't verified the release date. |
| Objectivity | 20% | A judgment by Priors, not a measurement. Some benchmarks have their own hand-set value. The rest default by grading method: 80 for percent scores, 65 for other scores, 45 for Elo, 40 for human or LLM judges. |
| Breadth | 15% | How many models it has measured, with full credit at 40. |
| Frontier coverage | 15% | The share of today's 10 default lineup models it has measured. |
Then points are added or taken away, and the total stays between 0 and 100:
- +15 for the benchmarks Priors prefers: DeepSWE v1.1, Terminal-Bench v4-AA, ARC-AGI-3, SimpleBench, BullshitBench v2, HLE-Diamond, AA-Omniscience v1, FrontierMath Tier 4 (v2), ProgramBench v1 Fully Resolved.
- −25 for a few older versions with a clear replacement.
- −20 if its results were last updated before 2026-09-22, when the current frontier models began to ship.
Quality doesn’t prove a benchmark is valid or free of training contamination, and part of it is our judgment. The same score guides the benchmarks suggested in the ranking flow and the builder.
Recommended
The directory’s default order: quality, lifted by how many published Priors pages give the benchmark weight.
quality + 12 × log2(1 + pages using it)
One page adds 12 points and three add 24. Each doubling adds another 12, so the first few pages count most.
The other columns
- Top model: the best score the benchmark reports, older models included.
- Models: how many the publisher has measured. Priors’ estimates don’t count.
- Mixes: how many published Priors pages give it weight.