Benchmarks · reasoning
BullshitBench v2
Share of 100 nonsense prompts (software, finance, legal, medical, physics) where the model clearly calls out the nonsense instead of confidently answering; 3-judge panel, mean aggregation. Reported per reasoning level.
108 models · 198 results · data as of 2026-10-29 · petergpt.github.io ↗
Leaderboard
Opus 4.8 none
All results
198 rows| # | Model | Effort | BullshitBench v2 |
|---|---|---|---|
| 1 | Opus 4.8 | none | 95.0% |
| 2 | Qwen 3.8 Max | minimal | 95.0% |
| 3 | Opus 4.8 | xhigh | 94.0% |
| 4 | Sonnet 4.6 | high | 91.0% |
| 5 | Opus 4.5 | high | 90.0% |
| 6 | Sonnet 4.6 | none | 89.0% |
| 7 | Opus 4.6 | high | 87.0% |
| 8 | Opus 4.6 | none | 83.0% |
| 9 | Opus 4.7 | none | 83.0% |
| 10 | Sonnet 5 | low | 80.0% |
| 11 | Mistral Large 4.0 | none | 80.0% |
| 12 | Opus 4.5 | none | 79.0% |
| 13 | Sonnet 4.5 | high | 79.0% |
| 14 | Qwen 3.8 27B | none | 79.0% |
| 15 | Sonnet 5 | max | 78.0% |
| 16 | Qwen 3.5 397B-A17B | high | 78.0% |
| 17 | Fable 5.1 | low | 77.0% |
| 18 | Haiku 4.5 | high | 77.0% |
| 19 | Opus 4.7 | max | 74.0% |
| 20 | Sonnet 4.5 | none | 74.0% |
| 21 | Kimi K3 | xhigh | 74.0% |
| 22 | Opus 5 | low | 73.0% |
| 23 | GLM 5.3 | minimal | 72.0% |
| 24 | Kimi K3 | minimal | 72.0% |
| 25 | Qwen 3.6 Plus | none | 72.0% |
| 26 | Qwen 3.7 Max | none | 72.0% |
| 27 | Haiku 4.5 | none | 71.0% |
| 28 | Opus 5 | xhigh | 70.0% |
| 29 | GPT-6 Astra | max | 69.0% |
| 30 | Qwen 3.5 397B-A17B | none | 69.0% |