Benchmarks · coding
FrontierCode 1.1-AA
AA's run of Cognition's FrontierCode 1.1 Main (100 private tasks, three attempts each, 250-turn cap): would a repository's maintainers merge the change? Mean score; an attempt failing a blocking check scores 0. Separate from Cognition's own leaderboard, which uses different harnesses.
25 models · 62 results · data as of 2026-10-29 · artificialanalysis.ai ↗
Leaderboard
Opus 5.5 medium
More from this benchmark
4 published measuresScore vs any published measure4 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||
|---|---|---|---|---|
| Opus 5.5 medium, fallback | 106,287,669 | 1,272,471 | 377,249 | 101,655,804 |
| GPT-6.1 Sol max | 195,193,427 | 1,253,375 | 2,066,048 | 184,538,167 |
| Fable 5.1 max, fallback | 1,480,245,965 | 4,235,752 | 5,183,665 | 1,458,954,633 |
| Gemini 4 Carbon high | 1,369,399,483 | 2,376,530 | 6,713,404 | 1,341,777,181 |
| GPT-6 Astra max | 167,225,761 | 1,139,367 | 1,552,657 | 157,858,375 |
| Sonnet 5.5 xhigh, fallback | 412,351,306 | 2,216,192 | 2,127,897 | 401,599,120 |
| Haiku 5.5 xhigh | 791,293,557 | 3,163,096 | 6,976,271 | 774,090,134 |
| MiMo V2.6 Pro | 737,882,657 | 1,851,973 | 6,588,853 | 724,053,179 |
| DeepSeek V4.1 Flash max | 2,243,262,295 | 2,913,666 | 8,962,309 | 2,220,427,419 |
| GPT-6.1 Terra medium | 128,783,916 | 899,601 | 342,013 | 123,164,294 |
| DeepSeek V4.1 Flash 1017 max | 1,645,238,967 | 2,800,799 | 5,684,876 | 1,626,961,579 |
| Kimi K3 max | 746,892,659 | 2,212,641 | 3,436,372 | 736,097,948 |
| Gemini 4 Argon high | 1,657,588,521 | 1,784,077 | 9,241,407 | 1,625,614,528 |
| MiMo V2.6 Flash | 860,121,596 | 1,846,704 | 7,601,950 | 844,487,010 |
| GPT-6 Luna max | 495,049,804 | 1,096,711 | 3,739,809 | 482,096,692 |
| GLM 5.3 Flash max | 1,037,506,529 | 2,045,313 | 4,090,197 | 1,024,897,873 |
| Muse Spark 1.3 max | 1,171,611,703 | 2,371,473 | 3,286,416 | 1,155,096,835 |
| GLM 5.3 max | 1,419,067,666 | 2,492,536 | 5,852,295 | 1,402,887,787 |
| Grok 4.7 low | 351,390,220 | 1,309,404 | 1,761,851 | 342,258,109 |
| Gemini 3.8 Flash medium | 1,664,264,087 | 1,882,819 | 6,411,349 | 1,642,518,096 |
| Qwen 3.8 27B medium | 855,588,978 | 2,223,532 | 4,673,021 | 842,735,544 |
| MiniMax M3 | 1,347,028,684 | 2,998,093 | 1,305,198 | 1,335,446,716 |
| Inkling xhigh | 1,102,032,487 | 1,294,031 | 2,902,252 | 1,088,507,694 |
| gpt-oss-120b high | 284,162,952 | 3,714,116 | 9,064,930 | 279,488,872 |
| Inkling Small | 446,919,381 | 1,849,662 | 3,724,508 | 441,335,872 |
All results
62 rows| # | Model | Effort | FrontierCode 1.1-AA | Harness |
|---|---|---|---|---|
| 1 | Opus 5.5 | medium | 57.1% | Stirrup |
| 2 | Opus 5.5 | high | 54.0% | Stirrup |
| 3 | Opus 5.5 | xhigh | 53.5% | Stirrup |
| 4 | Opus 5.5 | max | 52.2% | Stirrup |
| 5 | GPT-6.1 Sol | max | 49.1% | Stirrup |
| 6 | Fable 5.1 | max | 49.0% | Stirrup |
| 7 | Gemini 4 Carbon | high | 48.5% | Stirrup |
| 8 | GPT-6 Astra | max | 47.0% | Stirrup |
| 9 | GPT-6.1 Sol | high | 46.6% | Stirrup |
| 10 | GPT-6.1 Sol | xhigh | 46.3% | Stirrup |
| 11 | Sonnet 5.5 | xhigh | 46.1% | Stirrup |
| 12 | Sonnet 5.5 | max | 45.8% | Stirrup |
| 13 | Haiku 5.5 | xhigh | 45.5% | Stirrup |
| 14 | MiMo V2.6 Pro | 45.4% | Stirrup | |
| 15 | Sonnet 5.5 | high | 44.3% | Stirrup |
| 16 | DeepSeek V4.1 Flash | max | 44.0% | Stirrup |
| 17 | GPT-6.1 Terra | medium | 43.7% | Stirrup |
| 18 | DeepSeek V4.1 Flash 1017 | max | 43.7% | Stirrup |
| 19 | Haiku 5.5 | high | 43.6% | Stirrup |
| 20 | Kimi K3 | max | 43.6% | Stirrup |
| 21 | Gemini 4 Argon | high | 43.5% | Stirrup |
| 22 | GPT-6.1 Sol | medium | 43.3% | Stirrup |
| 23 | Opus 5.5 | low | 42.7% | Stirrup |
| 24 | MiMo V2.6 Flash | 42.1% | Stirrup | |
| 25 | GPT-6 Luna | max | 41.9% | Stirrup |
| 26 | Haiku 5.5 | max | 41.8% | Stirrup |
| 27 | GPT-6.1 Terra | xhigh | 41.4% | Stirrup |
| 28 | GPT-6.1 Terra | none | 41.0% | Stirrup |
| 29 | GLM 5.3 Flash | max | 41.0% | Stirrup |
| 30 | DeepSeek V4.1 Flash | none | 40.8% | Stirrup |