Benchmarks · math
BrokenArXiv 08/2026 (MathArena)
Plausible but false statements taken from August 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
14 models · 16 results · data as of 2026-10-29 · matharena.ai ↗
Leaderboard
GPT-6.1 Sol max
All results
16 rows| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-6.1 Sol | max | 86.3% |
| 2 | GPT-6 Astra | max | 81.9% |
| 3 | Fable 5.1 | high | 79.8% |
| 4 | GPT-6 Sol | max | 78.6% |
| 5 | Fable 5.1 | low | 78.0% |
| 6 | Opus 5.5 | high | 77.4% |
| 7 | GPT-6 Astra | low | 74.7% |
| 8 | Qwen 3.8 Max | max | 69.6% |
| 9 | Kimi K3.1 | max | 64.3% |
| 10 | Kimi K3 | max | 61.9% |
| 11 | DeepSeek V4.1 Flash | max | 54.8% |
| 12 | Gemini 4 Argon | max | 54.2% |
| 13 | Grok 4.7 | xhigh | 35.1% |
| 14 | Gemini 3.8 Flash | max | 27.4% |
| 15 | Muse Spark 1.3 | max | 20.2% |
| 16 | Mistral Large 4 | high | 11.3% |