Benchmarks · coding
SWE Atlas Test Writing (Scale AI)
Whether a coding agent writes production-grade tests for real repositories, graded with rubrics and LLM judges. Maintained by Scale AI. Score: Leaderboard score (percent); ± is the published confidence half-width.
23 models · 23 results · data as of 2026-10-29 · labs.scale.com ↗
Leaderboard
Fable 5.1 max
All results
23 rows| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Fable 5.1 | max | 67.0% |
| 2 | Opus 5 | xhigh | 62.2% |
| 3 | Fable 5 | max | 55.6% |
| 4 | Gemini 3.8 Flash | max | 53.7% |
| 5 | GPT-6 Astra Codex | xhigh | 50.7% |
| 6 | Opus 4.8 | xhigh | 49.6% |
| 7 | GPT-5.6 Sol Codex | max | 45.9% |
| 8 | GPT-5.4 Codex | xhigh | 44.4% |
| 9 | GPT-5.5 Codex | xhigh | 42.6% |
| 10 | GLM 5.2 | max | 41.5% |
| 11 | Muse Spark 1.1 | xhigh | 41.5% |
| 12 | GPT-5.4 Mini | xhigh | 40.0% |
| 13 | GPT-5.3 Codex | xhigh | 39.0% |
| 14 | Opus 4.7 | max | 38.5% |
| 15 | Opus 4.6 | max | 36.7% |
| 16 | Sonnet 4.6 | max | 31.8% |
| 17 | Muse Spark | 31.1% | |
| 18 | Gemini 3 Flash | high | 30.3% |
| 19 | Gemini 3.1 Pro | high | 29.8% |
| 20 | GLM 5 | high | 28.7% |
| 21 | DeepSeek V4 Pro | max | 27.1% |
| 22 | Kimi K2.5 | max | 25.8% |
| 23 | MiniMax M2.5 | high | 18.6% |