Benchmarks · reasoning
Chess LLM Benchmark
LLMs play rated chess games in a pool anchored by chess engines and Maia bots; the value is the Glicko rating, with a FIDE-scale estimate in details, for models with 10+ games.
53 models · 77 results · data as of 2026-10-31 · chessbenchllm.onrender.com ↗
Leaderboard
Gemini 3.1 Pro high
More from this benchmark
8 published measuresScore vs any published measure8 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | |||
|---|---|---|---|
| Gemini 3.1 Pro high | 2,007 | 18.0 | 2 |
| GPT-6 Astra high | 1,741 | 50.0 | 5 |
| Gemini 4 Argon medium | 1,738 | 36.0 | 6 |
| Gemini 3 Pro high | 1,730 | 51.0 | 7 |
| Gemini 3.5 Flash medium | 1,726 | 11.0 | 8 |
| Gemini 3.6 Flash medium | 1,692 | 30.0 | 9 |
| Gemini 3 Flash medium | 1,630 | 27.0 | 11.0 |
| GPT-3.5 high | 118.0 | 18.0 | |
| GPT-5.6 Sol high | 23.0 | 19.0 | |
| GPT-5.6 Luna xhigh | 17.0 | 21.0 | |
| GPT-5.1 high | 39.0 | 24.0 | |
| DeepSeek V4 Flash 0731 high | 13.0 | 27.0 | |
| o3 high | 27.0 | 28.0 | |
| Gemini 3.8 Flash medium | 10.0 | 31.0 | |
| Grok 4 Fast none | 52.0 | 32.0 | |
| GPT-5.6 Terra xhigh | 18.0 | 33.0 | |
| GPT-5.4 high | 30.0 | 39.0 | |
| GPT-5.2 high | 62.0 | 42.0 | |
| DeepSeek V4 Flash max | 15.0 | 45.0 | |
| gpt-oss-120b high | 52.0 | 47.0 | |
| GPT-5.5 low | 47.0 | 58.0 | |
| Gemini 2.5 Flash medium | 287.0 | 70.0 | |
| Gemini 2.5 Pro medium | 48.0 | 71.0 | |
| Opus 4.1 high | 202.0 | 74.0 | |
| Opus 4 medium | 202.0 | 75.0 |
All results
77 rows| # | Model | Effort | Chess LLM Benchmark |
|---|---|---|---|
| 1 | Gemini 3.1 Pro | high | 2144 ±181 |
| 2 | Gemini 3.1 Pro | medium | 2028 ±109 |
| 3 | GPT-6 Astra | high | 1975 ±106 |
| 4 | Gemini 4 Argon | medium | 1971 ±117 |
| 5 | Gemini 3 Pro | high | 1953 ±149 |
| 6 | Gemini 3.5 Flash | medium | 1944 ±187 |
| 7 | Gemini 3.6 Flash | medium | 1916 ±174 |
| 8 | Gemini 3 Flash | medium | 1857 ±170 |
| 9 | GPT-6 Astra | medium | 1779 ±196 |
| 10 | Gemini 3 Flash | high | 1750 ±250 |
| 11 | GPT-3.5 | high | 1586 ±128 |
| 12 | GPT-5.6 Sol | high | 1532 ±158 |
| 13 | GPT-5.6 Luna | xhigh | 1476 ±189 |
| 14 | GPT-5.6 Sol | xhigh | 1446 ±176 |
| 15 | GPT-5.1 | high | 1428 ±138 |
| 16 | GPT-5.6 Sol | medium | 1413 ±171 |
| 17 | DeepSeek V4 Flash 0731 | high | 1381 ±188 |
| 18 | o3 | high | 1334 ±255 |
| 19 | DeepSeek V4 Flash 0731 | max | 1294 ±196 |
| 20 | Gemini 3.8 Flash | medium | 1271 ±292 |
| 21 | Grok 4 Fast | none | 1267 ±130 |
| 22 | GPT-5.6 Terra | xhigh | 1210 ±181 |
| 23 | GPT-5.6 Sol | low | 1203 ±162 |
| 24 | GPT-5.4 | high | 1132 ±178 |
| 25 | GPT-5.2 | high | 1039 ±167 |
| 26 | DeepSeek V4 Flash | max | 917 ±238 |
| 27 | gpt-oss-120b | high | 681 ±140 |
| 28 | GPT-5.6 Terra | medium | 674 ±224 |
| 29 | GPT-5.2 | xhigh | 657 ±133 |
| 30 | Gemini 3 Flash | low | 650 ±132 |