Benchmarks · reasoning
HELM Lite
Uses HELM Lite's core-scenarios Accuracy table mean win rate. Score: core-scenarios mean win rate. Collected by Jake Boggs' benchmark dashboard.
40 models · 40 results · data as of 2026-10-29 · crfm.stanford.edu ↗
Leaderboard
GPT-4o high
All results
40 rows| # | Model | Effort | HELM Lite |
|---|---|---|---|
| 1 | GPT-4o | high | 93.8% |
| 2 | DeepSeek V3 | none | 90.8% |
| 3 | Sonnet 3.5 | medium | 88.5% |
| 4 | GPT-4 | high | 86.7% |
| 5 | Llama 3.1 | max | 85.4% |
| 6 | Gemini 1.5 Pro | none | 84.2% |
| 7 | Llama 3.2 | 81.9% | |
| 8 | Gemini 2.0 Flash | medium | 81.3% |
| 9 | Llama 3.3 | none | 81.2% |
| 10 | Llama 3 70b | none | 79.3% |
| 11 | Qwen 2 72B | none | 77.0% |
| 12 | Qwen 2.5 Turbo | none | 74.5% |
| 13 | Mistral Large 2 | none | 74.4% |
| 14 | GPT-4o mini | high | 70.1% |
| 15 | Opus 3 | none | 68.3% |
| 16 | Gemma 2 27B | 67.5% | |
| 17 | Gemini 1.5 Flash | none | 66.7% |
| 18 | Jamba 1.5 Large | none | 63.7% |
| 19 | Qwen 1.5 72B | 60.8% | |
| 20 | Gemma 2 9B | 56.2% | |
| 21 | Qwen 1.5 110B | none | 55.0% |
| 22 | Qwen 1.5 32B | 54.6% | |
| 23 | Haiku 3.5 | medium | 53.1% |
| 24 | Llama 2 70b | none | 48.2% |
| 25 | GPT-3.5 | high | 43.9% |
| 26 | Qwen 1.5 14B | 42.5% | |
| 27 | Gemini 1.0 Pro | none | 42.2% |
| 28 | Jamba 1.5 Mini | none | 41.4% |
| 29 | Llama 3 8b | none | 38.7% |
| 30 | Sonnet 3 | none | 37.7% |