Benchmarks · knowledge
MMLU Pro (Vals)
Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences. Scores are the values published by VALS AI. Source benchmark version 1.
56 models · 56 results · updated 2026-09-01 · vals.ai ↗
Leaderboard
Fable 5.1 max
More from this benchmark
15 published measuresTime per task137 results · lower is better
Score vs any published measure14 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Fable 5.1 max, fallback | 96.1 | 95.2 | 93.8 | 94.9 | 95.1 | 90.8 | 86.7 | 85.3 | 87.5 | 97.3 | 89.9 | 90.2 | 96.7 | 93.7 |
| Opus 5 max | 95.7 | 93.5 | 93.3 | 93.9 | 93.7 | 89.9 | 87.8 | 86.6 | 86.6 | 97.1 | 88.0 | 89.0 | 94.8 | 92.5 |
| Fable 5 max | 93.6 | 93.2 | 93.9 | 94.1 | 93.8 | 90.0 | 84.6 | 85.3 | 86.5 | 97.6 | 89.8 | 90.6 | 95.8 | 92.2 |
| Gemini 3.8 Flash high | 95.0 | 93.8 | 92.5 | 92.7 | 92.1 | 88.8 | 84.5 | 83.7 | 82.0 | 96.2 | 87.6 | 88.2 | 94.8 | 91.5 |
| Gemini 3.7 Flash high | 94.7 | 92.8 | 92.3 | 92.0 | 91.7 | 89.3 | 84.7 | 84.0 | 81.9 | 96.0 | 87.6 | 89.4 | 94.3 | 91.1 |
| Opus 4.7 max | 94.6 | 93.3 | 93.3 | 92.7 | 92.3 | 88.0 | 82.4 | 83.2 | 82.1 | 95.9 | 86.9 | 89.0 | 93.4 | 91.2 |
| Opus 4.8 max | 94.7 | 92.6 | 92.8 | 92.7 | 91.8 | 87.7 | 85.1 | 82.7 | 81.9 | 95.9 | 86.1 | 86.6 | 93.5 | 90.0 |
| Gemini 3.5 Flash high | 93.9 | 93.3 | 93.1 | 91.7 | 91.9 | 87.8 | 83.6 | 81.6 | 80.4 | 96.3 | 87.6 | 87.6 | 94.1 | 90.4 |
| Grok 4.6 high | 95.0 | 92.6 | 92.0 | 91.7 | 91.8 | 85.8 | 84.4 | 81.1 | 81.8 | 95.9 | 87.2 | 89.2 | 92.1 | 90.9 |
| Qwen 3.7 Max | 95.1 | 93.5 | 92.3 | 91.2 | 91.7 | 85.8 | 84.4 | 82.2 | 79.5 | 96.0 | 86.0 | 89.2 | 93.5 | 90.0 |
| Gemini 3.6 Flash high | 94.7 | 92.8 | 92.3 | 91.7 | 90.5 | 86.7 | 84.6 | 83.5 | 80.1 | 96.0 | 86.5 | 87.6 | 93.5 | 89.5 |
| Grok 4.5 high | 94.3 | 93.3 | 91.6 | 91.0 | 91.4 | 87.2 | 84.0 | 81.4 | 82.0 | 95.9 | 87.2 | 87.6 | 92.0 | 90.2 |
| GPT-5.6 Sol max | 94.6 | 93.3 | 92.0 | 90.7 | 90.6 | 87.8 | 84.1 | 81.9 | 79.1 | 95.3 | 87.7 | 88.4 | 92.8 | 89.2 |
| Muse Spark 1.1 xhigh | 95.0 | 94.5 | 93.4 | 90.7 | 91.2 | 88.1 | 83.4 | 79.8 | 76.5 | 96.0 | 85.3 | 87.2 | 93.8 | 87.2 |
| Qwen 3.8 Max | 94.1 | 92.9 | 92.5 | 91.0 | 89.9 | 86.7 | 84.0 | 81.1 | 77.3 | 95.5 | 85.3 | 88.4 | 93.3 | 88.5 |
| Muse Spark 1.2 xhigh | 93.9 | 92.0 | 92.8 | 90.7 | 91.2 | 86.3 | 82.0 | 81.9 | 76.7 | 96.0 | 85.0 | 87.4 | 93.4 | 86.6 |
| GPT-5.5 xhigh | 94.0 | 92.0 | 91.3 | 89.8 | 89.9 | 84.8 | 84.4 | 81.6 | 76.9 | 94.7 | 86.8 | 87.8 | 91.5 | 88.5 |
| Kimi K3 max | 93.9 | 92.0 | 91.8 | 91.2 | 91.6 | 86.9 | 82.0 | 79.3 | 77.0 | 95.6 | 84.0 | 85.4 | 92.6 | 88.3 |
| Qwen 3.6 Plus max | 93.7 | 92.3 | 91.4 | 89.8 | 89.7 | 84.3 | 80.8 | 81.1 | 75.7 | 95.5 | 85.7 | 87.2 | 92.8 | 87.5 |
| Kimi K2.6 max | 93.7 | 91.6 | 92.5 | 91.2 | 89.9 | 87.1 | 80.7 | 78.5 | 74.3 | 95.6 | 85.0 | 84.6 | 92.1 | 89.2 |
| Sonnet 5 max | 93.6 | 91.6 | 90.8 | 89.0 | 89.5 | 81.6 | 82.3 | 80.3 | 78.7 | 95.9 | 85.4 | 86.6 | 91.5 | 88.7 |
| Sonnet 4.6 max | 92.7 | 91.9 | 91.9 | 90.0 | 90.6 | 85.2 | 82.4 | 81.1 | 73.6 | 94.1 | 83.1 | 86.2 | 92.6 | 87.3 |
| Muse Spark | 93.7 | 91.9 | 91.5 | 90.0 | 89.8 | 84.6 | 80.8 | 80.8 | 75.0 | 95.0 | 84.6 | 85.4 | 92.7 | 86.5 |
| MiniMax M2.1 | 93.6 | 90.1 | 91.3 | 87.1 | 90.3 | 81.8 | 83.7 | 79.5 | 78.9 | 92.7 | 86.0 | 86.0 | 89.5 | 88.0 |
| DeepSeek V4 Pro 0813 max | 93.7 | 91.0 | 91.7 | 90.0 | 89.2 | 83.4 | 81.3 | 78.5 | 74.9 | 95.1 | 84.5 | 84.6 | 92.5 | 87.2 |
All results
56 rows| # | Model | Effort | MMLU Pro (Vals) | Subscription $ / task | API $ / task | Time |
|---|---|---|---|---|---|---|
| 1 | Fable 5.1 | max | 92.4% | $0.015 | $0.098 | 21s |
| 2 | Opus 5 | max | 91.6% | $0.000 | $0.028 | 9.6s |
| 3 | Fable 5 | max | 91.5% | $0.009 | $0.082 | 25s |
| 4 | Gemini 3.8 Flash | high | 90.2% | $0.001 | $0.022 | 7.8s |
| 5 | Gemini 3.7 Flash | high | 90.1% | $0.001 | $0.011 | 4.2s |
| 6 | Opus 4.7 | max | 89.9% | $0.062 | $0.062 | 3.1 min |
| 7 | Opus 4.8 | max | 89.6% | $0.001 | $0.061 | 23s |
| 8 | Gemini 3.5 Flash | high | 89.5% | $0.025 | $0.025 | 26s |
| 9 | Grok 4.6 | high | 89.4% | $0.002 | $0.015 | 55s |
| 10 | Qwen 3.7 Max | max | 89.3% | $0.005 | $0.017 | 29s |
| 11 | Gemini 3.6 Flash | high | 89.3% | $0.000 | $0.013 | 7.3s |
| 12 | Grok 4.5 | high | 89.2% | $0.002 | $0.015 | 6.0 min |
| 13 | GPT-5.6 Sol | max | 89.1% | $0.002 | $0.032 | 16s |
| 14 | Muse Spark 1.1 | xhigh | 88.7% | $0.011 | $0.011 | 21s |
| 15 | Qwen 3.8 Max | max | 88.6% | $0.019 | $0.019 | 64s |
| 16 | Muse Spark 1.2 | xhigh | 88.3% | $0.005 | $0.011 | 44s |
| 17 | GPT-5.5 | xhigh | 88.1% | $0.002 | $0.028 | 42s |
| 18 | Kimi K3 | max | 88.0% | $0.002 | $0.021 | 39s |
| 19 | Qwen 3.6 Plus | max | 87.7% | $0.009 | $0.009 | 46s |
| 20 | Kimi K2.6 | max | 87.6% | $0.018 | $0.018 | 2.6 min |
| 21 | Sonnet 5 | max | 87.5% | $0.001 | $0.043 | 25s |
| 22 | Sonnet 4.6 | max | 87.3% | $0.066 | $0.066 | 55s |
| 23 | Muse Spark | 87.3% | $0.002 | $0.002 | 46s | |
| 24 | MiniMax M2.1 | 87.0% | $0.003 | $0.003 | 41s | |
| 25 | DeepSeek V4 Pro 0813 | max | 87.0% | $0.013 | $0.013 | 60s |
| 26 | GLM 5.1 | max | 86.9% | $0.012 | $0.012 | 69s |
| 27 | GLM 5.3 | max | 86.8% | $0.002 | $0.017 | 62s |
| 28 | GLM 5.2 | max | 86.7% | $0.001 | $0.014 | 52s |
| 29 | GPT-5.6 Terra | xhigh | 86.7% | $0.000 | $0.008 | 8.2s |
| 30 | Inkling | max | 86.3% | free | $0.040 | 73s |