Benchmarks · reasoning
SimpleQA Verified
1,000 short fact-seeking questions with human-verified answer keys (DeepMind's relabeling of OpenAI's SimpleQA); accuracy on the verified set.
49 models · 49 results · data as of 2026-10-29 · kaggle.com ↗
Leaderboard
Gemini 3.1 Pro high
All results
49 rows| # | Model | Effort | SimpleQA Verified |
|---|---|---|---|
| 1 | Gemini 3.1 Pro | high | 77.5% |
| 2 | GPT-6 Astra | max | 75.8% |
| 3 | Gemini 4 Argon | max | 75.6% |
| 4 | Gemini 3.8 Flash | max | 74.6% |
| 5 | GPT-6.1 Sol | max | 73.8% |
| 6 | Gemini 3.7 Flash | max | 73.2% |
| 7 | Gemini 3 Pro | high | 72.1% |
| 8 | Gemini 3.5 Flash | xhigh | 70.4% |
| 9 | GPT-5.6 Sol | max | 69.2% |
| 10 | GPT-6 Sol | max | 64.4% |
| 11 | Opus 5 | max | 63.3% |
| 12 | Grok 4.7 | max | 59.4% |
| 13 | Grok 4.5 | max | 53.8% |
| 14 | Qwen 3 235B-A22B | xhigh | 53.7% |
| 15 | Grok 4.6 | max | 52.9% |
| 16 | o3 | high | 52.3% |
| 17 | Grok 4 | high | 51.9% |
| 18 | Opus 4.6 | max | 45.2% |
| 19 | Sonnet 5.5 | max | 44.7% |
| 20 | Opus 4.8 | max | 44.5% |
| 21 | DeepSeek V4.1 Flash | max | 42.3% |
| 22 | GPT-5.6 Terra | max | 41.5% |
| 23 | Grok 3 | none | 39.3% |
| 24 | GPT-5.6 Luna | max | 39.1% |
| 25 | Opus 4.5 | high | 39.0% |
| 26 | GPT-5.1 | high | 34.9% |
| 27 | Gemini 3.5 Flash-Lite | xhigh | 33.1% |
| 28 | Sonnet 5 | max | 32.1% |
| 29 | Opus 4.1 | high | 31.0% |
| 30 | Sonnet 4.6 | max | 30.9% |