Benchmarks · reasoning
OpenBookQA
Uses Epoch AI's external OpenBookQA accuracy leaderboard. Random baseline is 25% for four-way multiple choice. Score: OpenBookQA accuracy. Collected by Jake Boggs' benchmark dashboard.
12 models · 12 results · data as of 2026-10-29 · epoch.ai ↗
Leaderboard
GPT-3.5 high
All results
12 rows| # | Model | Effort | OpenBookQA |
|---|---|---|---|
| 1 | GPT-3.5 | high | 86.0% |
| 2 | Mixtral 8x7B Instruct | none | 85.8% |
| 3 | Llama 3 8b | none | 82.6% |
| 4 | Mistral | none | 79.8% |
| 5 | Llama 2 70b | none | 60.2% |
| 6 | Llama 6 5b | 60.2% | |
| 7 | Llama 2 7b | none | 58.6% |
| 8 | Llama 3 3b | 58.6% | |
| 9 | Llama 2 34b | 58.2% | |
| 10 | Llama 2 13b | none | 57.0% |
| 11 | Llama 1 3b | 56.4% | |
| 12 | GPT-2 | 22.4% |