Scoring methods
A mix is a set of benchmarks with weights. A scoring method turns a model’s results on that mix into one number. Priors has three. The weighted average is the default and what every ranking uses unless you change it under Calculation settings in the mix editor. A ranking that uses another method says so beside its scores.
All three rank the same models. Whether a model has enough measured data to be ranked is decided first, by the same rules, so switching method never adds or removes a model.
Weighted average
Each benchmark’s score, on a 0 to 100 scale, times its weight, added up. Percent scores are used as published. A benchmark at 15% contributes 15% of the score.
When a model has no result on a benchmark, the gap is filled with an estimate from similar models, so a model is not rewarded for skipping its weakest benchmarks.
Limits. It treats a point on an easy benchmark and a point on a hard one as the same thing. Going from 90 to 95 on a nearly solved benchmark counts as much as going from 10 to 15 on one almost nobody can do. And the filled gaps are guesses that look as exact as measurements.
IRT ability
Item response theory comes from exam scoring. It assumes each model has one underlying ability and each benchmark has a difficulty and a steepness: how much ability it takes to score halfway, and how quickly scores rise past that point. Priors works out those two numbers for every benchmark from all the models that took it, then finds the ability that best explains a model’s results on your mix.
How an ability becomes a 0 to 100 score
An ability is a position along the bottom of that chart: a number like 0.75, with no top or bottom of its own. Priors does not stretch or rescale it onto 0 to 100. It turns it back into benchmark scores, in two steps.
- Read every benchmark’s curve at the model’s ability. That gives the score the curve expects from a model that able.
- Average those expected scores with your weights. This is the same sum the weighted average does, on expected scores instead of real ones.
That is why the score cannot leave 0 to 100. Every reading is a benchmark score, so it lies between 0 and 100, and an average of them does too. A score of 100 would mean the curves expect full marks on every benchmark in the mix. It is also why the score reads like the default one: it is the same weighted average, with each real score replaced by what the model’s ability predicts. The likely range is converted the same way, by reading the curves at both ends of the ability’s range.
Your weights also decide how the ability itself is found. A benchmark’s weight is its share of the pull on ability, so a steep benchmark does not pull harder than its weight says.
When it differs. A model that does well on hard benchmarks and loses a few points on easy ones moves up. A model whose average is carried by one unusual result moves down, because one ability cannot explain an outlier. Nothing is filled in: a benchmark a model never ran is left to its curve.
Limits. One ability per model is a simplification, so a model that is strong at one kind of task and weak at another is described less well. The ability starts from how the model does across every benchmark Priors has, not only your mix, so on a mix of one or two benchmarks it follows them less closely than the average does. Each score has a likely range, and the ranges of neighbouring models usually overlap.
ECI-style
Epoch AI publishes the Epoch Capabilities Index: one capability number per model, fitted across many benchmarks at once. Each benchmark gets a difficulty and a slope, each model a capability, and a model’s expected score on a benchmark is sigmoid(slope × (capability − difficulty)). Capabilities are then rescaled in a straight line onto a scale with no upper limit.
This method runs the same kind of fit on the benchmarks in your mix. It is not Epoch’s index, which Priors also carries, unchanged, as the benchmark Epoch Capabilities Index. It differs in these ways:
- Benchmarks. Epoch fits over 50 benchmarks it chose. This uses only the ones in your mix, including Elo ratings converted to 0 to 100, which Epoch leaves out.
- Weights. Epoch has none. Here a benchmark’s errors count in proportion to its weight, so a benchmark at 20% pulls twice as hard on capability as one at 10%.
- Fit. Since November 4, 2026 Epoch samples a Bayesian version of the model, which takes minutes. This uses the method Epoch used until then, least squares with a small penalty on every parameter, because it has to run while you drag a slider.
- Scale. Epoch pins Claude 3.5 Sonnet at 130 and GPT-5 at 150. Neither has results on today’s benchmarks, so the scale here is lined up with Epoch’s published scores instead: over the models both score, the average and the spread are made to match. If a ranking has fewer than five such models, the middle model reads 150.
- Which models. Epoch scores a model with at least four benchmark results in the middle of the range. Priors keeps its own coverage rule, the same one the other methods use.
- Ranges. Epoch reports 90% intervals from its samples. Priors shows a 95% range worked out from how well the fit explains each model’s scores.
When it differs. In the same direction as IRT ability: results on hard benchmarks count for more than the same number of points on easy ones. Unlike IRT it uses your mix alone, and it reports capability itself, so the numbers are not 0 to 100.
Limits. Epoch says of its own index that a score means little by itself and is for comparing models. That holds here too. A model with scores near zero on everything gets a very low number with a very wide range. With one benchmark in the mix it can only repeat that benchmark’s order.
Why one stops at 100 and the other does not
Both methods begin the same way: a fit that places every model at a position on one axis. They differ in what they report from that position.
- IRT ability reports a predicted score. It reads the benchmark curves at the model’s position and averages them. Every curve flattens as it nears 100, so the score does too.
- ECI-style reports the position itself. It puts the position through a straight line onto Epoch’s scale. A straight line never flattens, so the score has no ceiling.
64.2153.13 + 27.39 × 0.628 = 170.3So the same gap between two strong models shows in full on the ECI-style scale and shrinks on the IRT ability scale as the mix gets closer to solved. Neither is more correct. A 0 to 100 score says how much of this mix a model is expected to get. An open scale keeps separating models after that, though with wider ranges, because a nearly solved benchmark says little about how much further a model could go.
The same mix, three ways
The top 10 on the General frontier mix as of 2026-11-22, each model at its best reasoning effort, with its likely range underneath.
| # | Weighted average | IRT ability | ECI-style |
|---|---|---|---|
| 1 | Fable 5.2 max 64.5 | Gemini 4 Neon xhigh 64.256.1 to 71.0 | Fable 5.2 max 171.8169.1 to 174.5 |
| 2 | GPT-6.2 Astra max 62.1 | Fable 5.2 max 62.854.8 to 69.8 | GPT-6.2 Astra max 171.5168.9 to 174.1 |
| 3 | Gemini 4 Neon xhigh 61.4 | Gemini 4 Carbon high 62.754.7 to 69.7 | Gemini 4 Neon xhigh 170.3167.9 to 172.8 |
| 4 | Gemini 4 Carbon high 60.0 | GPT-6.2 Astra max 60.252.1 to 67.4 | Gemini 4 Carbon high 169.7167.4 to 172.1 |
| 5 | Opus 5.5 max 58.9 | Opus 5.5 max 58.550.5 to 65.9 | GPT-6.2 Sol max 169.6167.2 to 171.9 |
| 6 | GPT-6.2 Sol max 57.3 | Gemini 4 Hydrogen xhigh 57.349.3 to 64.7 | Opus 5.5 max 169.3167.0 to 171.6 |
| 7 | GPT-6 Astra max 56.9 | GPT-6 Astra max 56.148.1 to 63.6 | GPT-6 Astra max 169.2166.9 to 171.5 |
| 8 | Sonnet 5.5 max 54.6 | GPT-6.2 Sol max 55.847.9 to 63.4 | GPT-6.1 Sol max 168.3166.1 to 170.5 |
| 9 | GPT-6.1 Sol max 54.6 | Sonnet 5.5 max 54.646.7 to 62.2 | Sonnet 5.5 max 167.9165.7 to 170.1 |
| 10 | Gemini 4 Hydrogen xhigh 53.9 | GPT-6.1 Sol max 53.745.8 to 61.3 | GPT-6.1 Terra max 167.5165.3 to 169.7 |
Open this mix with IRT ability or ECI-style scores.
What stays the same
- Benchmark scores are used as published under every method.
- Cost, speed and time per task do not depend on the method.
- Rankings through time always use the weighted average, so one day compares with the next.
- The method travels with the ranking: in its link, on a saved page and on its share card.