Benchmarks · coding
SWE-bench Verified (Vals)
Solving production software engineering tasks Scores are the values published by VALS AI. Source benchmark version 1.
53 models · 53 results · updated 2026-09-01 · vals.ai ↗
Leaderboard
Opus 5 max
More from this benchmark
5 published measuresTime per task87 results · lower is better
Score vs any published measure4 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||
|---|---|---|---|---|
| Opus 5 max | 90.5 | 96.9 | 98.5 | 100.0 |
| DeepSeek V4 Pro 0813 max | 100.0 | 95.0 | 97.4 | 100.0 |
| GPT-5.6 Sol max | 97.6 | 95.0 | 97.4 | 100.0 |
| Grok 4.6 high | 92.9 | 95.8 | 95.9 | 100.0 |
| GLM 5.3 max | 90.5 | 94.6 | 97.4 | 100.0 |
| GPT-5.6 Terra max | 97.6 | 93.9 | 96.9 | 100.0 |
| Fable 5 max | 92.9 | 94.6 | 95.9 | 100.0 |
| Kimi K3 max | 92.9 | 94.6 | 92.3 | 66.7 |
| GPT-5.6 Luna max | 85.7 | 92.0 | 96.4 | 66.7 |
| GLM 5.3 Flash max | 83.3 | 92.3 | 93.3 | 100.0 |
| DeepSeek V4 Flash 0731 high | 90.5 | 87.4 | 90.7 | 66.7 |
| Opus 4.8 max | 73.8 | 88.1 | 92.8 | 66.7 |
| Grok 4.5 high | 73.8 | 83.5 | 93.8 | 66.7 |
| Muse Spark 1.2 xhigh | 64.3 | 86.6 | 92.3 | 33.3 |
| Qwen 3.8 27B xhigh | 69.0 | 85.8 | 90.2 | 66.7 |
| Qwen 3.8 Max | 64.3 | 83.5 | 93.3 | 66.7 |
| GLM 5.2 max | 66.7 | 81.6 | 88.1 | 66.7 |
| GPT-5.5 xhigh | 50.0 | 81.2 | 91.8 | 66.7 |
| Inkling Small max | 42.9 | 80.5 | 93.8 | 33.3 |
| Opus 4.7 max | 64.3 | 78.9 | 90.2 | 66.7 |
| Muse Spark 1.1 xhigh | 52.4 | 82.4 | 88.1 | 66.7 |
| Gemini 3.7 Flash high | 57.1 | 80.5 | 87.1 | 33.3 |
| Gemini 3.8 Flash high | 52.4 | 77.8 | 89.7 | 33.3 |
| Sonnet 5 max | 76.2 | 77.4 | 83.5 | 66.7 |
| Composer 2.5 none | 52.4 | 78.9 | 86.6 | 66.7 |
All results
53 rows| # | Model | Effort | Score | Subscription $ / task | API $ / task | Time | Harness |
|---|---|---|---|---|---|---|---|
| 1 | Opus 5 | max | 97.0% | $0.022 | $1.29 | 9.6 min | |
| 2 | DeepSeek V4 Pro 0813 | max | 96.4% | $0.10 | $0.10 | 4.0 min | |
| 3 | GPT-5.6 Sol | max | 96.2% | $0.064 | $1.15 | 3.0 min | |
| 4 | Grok 4.6 | high | 95.6% | $0.081 | $0.78 | 10 min | |
| 5 | GLM 5.3 | max | 95.4% | $0.035 | $0.34 | 14 min | |
| 6 | GPT-5.6 Terra | max | 95.4% | $0.022 | $0.40 | 3.0 min | |
| 7 | Fable 5 | max | 95.0% | $0.23 | $2.05 | 5.9 min | |
| 8 | Kimi K3 | max | 93.4% | $0.059 | $0.76 | 10 min | |
| 9 | GPT-5.6 Luna | max | 93.0% | $0.004 | $0.043 | 3.4 min | |
| 10 | GLM 5.3 Flash | max | 92.0% | free | $0.018 | 1.1 h | |
| 11 | DeepSeek V4 Flash 0731 | high | 88.8% | $0.010 | $0.010 | 3.4 min | |
| 12 | Opus 4.8 | max | 88.6% | $0.033 | $1.92 | 9.4 min | |
| 13 | Grok 4.5 | high | 86.6% | $0.085 | $0.54 | 3.3 min | |
| 14 | Muse Spark 1.2 | xhigh | 86.6% | $0.28 | $0.55 | 8.1 min | |
| 15 | Qwen 3.8 27B | xhigh | 86.0% | $0.079 | $1.08 | 10.0 min | |
| 16 | Qwen 3.8 Max | max | 85.6% | $1.13 | $1.13 | 42 min | |
| 17 | GLM 5.2 | max | 82.8% | $0.062 | $0.71 | 11 min | |
| 18 | GPT-5.5 | xhigh | 82.6% | $0.075 | $1.36 | 7.1 min | |
| 19 | Inkling Small | max | 82.2% | free | $0.13 | 4.1 min | |
| 20 | Opus 4.7 | max | 82.0% | $2.42 | $2.42 | 7.4 min | |
| 21 | Muse Spark 1.1 | xhigh | 82.0% | $0.35 | $0.35 | 3.5 min | |
| 22 | Gemini 3.7 Flash | high | 80.8% | $0.19 | $1.44 | 5.8 min | |
| 23 | Gemini 3.8 Flash | high | 80.0% | $0.089 | $2.19 | 11 min | |
| 24 | Sonnet 5 | max | 79.6% | $0.026 | $1.49 | 16 min | |
| 25 | Composer 2.5 | none | 79.6% | Cursor CLI | |||
| 26 | Gemini 3.6 Flash | high | 79.6% | $0.038 | $1.19 | 13 min | |
| 27 | Gemini 3.5 Flash | high | 78.8% | $0.95 | $0.95 | 4.2 min | |
| 28 | Kimi K2.7 Code | max | 78.2% | $0.058 | $0.27 | 15 min | |
| 29 | GPT-5.3 Codex | xhigh | 78.0% | $0.46 | $0.46 | 4.1 min | |
| 30 | Inkling | max | 77.6% | free | $0.59 | 6.6 min |