Benchmarks · coding
DeepSWE v1
Original DeepSWE leaderboard: pass@1 on 113 long-horizon SWE tasks with mini-swe-agent. Used as fallback for models not yet run on v1.1.
21 models · 29 results · data as of 2026-10-29 · deepswe.datacurve.ai ↗
Leaderboard
GPT-5.5 xhigh
More from this benchmark
3 published measuresAgent steps per task29 results · lower is better
Time per task29 results · lower is better
Output tokens per task29 results · lower is better
All results
29 rows| # | Model | Effort | DeepSWE v1 | Subscription $ / task | API $ / task | Agent steps | Time | Output tokens | Harness |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-5.5 | xhigh | 70.0% ±2.8 | $0.36 | $6.61 | 79 | 21 min | 47k | mini-swe-agent |
| 2 | GPT-5.5 | high | 61.9% ±3.6 | $0.25 | $4.47 | 58 | 15 min | 31k | mini-swe-agent |
| 3 | Opus 4.8 | max | 58.2% ±2.3 | $0.22 | $13 | 108 | 43 min | 136k | mini-swe-agent |
| 4 | Opus 4.8 | xhigh | 57.7% ±4.0 | $0.13 | $7.55 | 87 | 32 min | 86k | mini-swe-agent |
| 5 | GPT-5.4 | xhigh | 55.5% ±2.2 | $4.38 | $4.38 | 69 | 27 min | 71k | mini-swe-agent |
| 6 | Opus 4.7 | max | 54.2% ±4.7 | $18 | $18 | 203 | 39 min | 103k | mini-swe-agent |
| 7 | Opus 4.8 | high | 50.7% ±3.0 | $0.069 | $3.98 | 66 | 20 min | 48k | mini-swe-agent |
| 8 | GPT-5.5 | medium | 48.0% ±2.9 | $0.13 | $2.34 | 42 | 9.8 min | 19k | mini-swe-agent |
| 9 | Opus 4.8 | medium | 47.3% ±3.9 | $0.056 | $3.25 | 60 | 19 min | 40k | mini-swe-agent |
| 10 | Opus 4.7 | xhigh | 44.7% ±2.9 | $8.58 | $8.58 | 126 | 24 min | 66k | mini-swe-agent |
| 11 | GLM 5.2 | max | 41.5% ±2.9 | $0.34 | $3.95 | 125 | 44 min | 82k | mini-swe-agent |
| 12 | Opus 4.7 | high | 40.3% ±3.5 | $4.83 | $4.83 | 87 | 17 min | 46k | mini-swe-agent |
| 13 | Sonnet 4.6 | high | 31.8% ±1.9 | $5.52 | $5.52 | 129 | 42 min | 76k | mini-swe-agent |
| 14 | Opus 4.7 | medium | 31.6% ±4.0 | $2.34 | $2.34 | 55 | 10 min | 27k | mini-swe-agent |
| 15 | Gemini 3.5 Flash | medium | 28.3% ±3.5 | $7.42 | $7.42 | 75 | 17 min | 189k | mini-swe-agent |
| 16 | Opus 4.6 | max | 27.6% ±3.7 | $5.39 | $5.39 | 103 | 30 min | 44k | mini-swe-agent |
| 17 | GPT-5.4 Mini | xhigh | 24.3% ±3.0 | $2.08 | $2.08 | 86 | 33 min | 135k | mini-swe-agent |
| 18 | Kimi K2.6 | max | 23.9% ±2.5 | $3.16 | $3.16 | 147 | 56 min | 84k | mini-swe-agent |
| 19 | MiniMax M3 | max | 20.4% ±3.7 | $0.94 | $5.57 | 314 | 57 min | 98k | mini-swe-agent |
| 20 | MiMo V2.5 Pro | xhigh | 19.5% ±1.9 | $0.13 | $1.99 | 121 | 28 min | 49k | mini-swe-agent |
| 21 | Qwen 3.7 Max | max | 17.7% ±1.4 | $0.64 | $2.12 | 110 | 17 min | 42k | mini-swe-agent |
| 22 | GLM 5.1 | max | 17.5% ±0.8 | $7.46 | $7.46 | 177 | 35 min | 49k | mini-swe-agent |
| 23 | Grok Build 0.1 | 13.1% ±2.4 | $6.60 | $6.60 | 176 | 44 min | 52k | mini-swe-agent | |
| 24 | Gemini 3.1 Pro | high | 9.7% ±2.8 | $1.84 | $1.84 | 74 | 36 min | 53k | mini-swe-agent |
| 25 | DeepSeek V4 Pro | max | 7.5% ±2.7 | $0.70 | $4.22 | 111 | 37 min | 50k | mini-swe-agent |
| 26 | Gemini 3 Flash | high | 5.1% ±2.5 | $1.53 | $1.53 | 71 | 39 min | 233k | mini-swe-agent |
| 27 | Qwen 3.6 Plus | max | 2.7% ±1.2 | $4.25 | $4.25 | 164 | 35 min | 67k | mini-swe-agent |
| 28 | Haiku 4.5 | xhigh | 0.2% ±0.3 | $0.84 | $0.84 | 109 | 9.2 min | 39k | mini-swe-agent |
| 29 | MiniMax M2.7 | high | 0.2% ±0.3 | $0.13 | $0.70 | 136 | 35 min | 60k | mini-swe-agent |