Benchmarks · agentic
AA-Briefcase v1.1
AA-Briefcase: agentic office-work tasks over real files (xlsx/pptx/docx/pdf) judged pairwise; reported as Elo.
86 models · 167 results · data as of 2026-10-29 · artificialanalysis.ai ↗
Leaderboard
Sonnet 5.5 max
More from this benchmark
43 published measuresScore vs any published measure43 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||||
|---|---|---|---|---|---|---|
| Sonnet 5.5 max, fallback | 1,843 | 5,202 | 16,795 | 13,505 | 9,030 | 7,403 |
| Opus 5.5 max, fallback | 1,274 | 3,308 | 9,093 | 7,144 | 5,235 | 4,158 |
| Fable 5.1 max, fallback | 625.0 | 2,010 | 5,302 | 5,383 | 3,430 | 2,209 |
| Opus 5 max | 937.0 | 2,274 | 5,349 | 5,347 | 4,100 | 1,978 |
| Grok 4.7 xhigh | 634.0 | 1,605 | 3,735 | 4,613 | 3,155 | 1,766 |
| Gemini 4 Carbon high | 448.0 | 2,341 | 2,488 | 3,059 | 2,444 | 406.0 |
| Muse Spark 1.3 max | 509.0 | 1,665 | 3,869 | 4,642 | 3,869 | 471.0 |
| Haiku 5.5 max | 1,234 | 1,153 | 8,021 | 5,002 | 3,948 | 5,309 |
| DeepSeek V4.1 Flash 1017 max | 881.0 | 2,393 | 6,988 | 6,971 | 5,605 | 3,040 |
| GPT-6 Astra max | 279.0 | 1,058 | 2,308 | 1,853 | 1,568 | 589.0 |
| GPT-6.1 Sol max | 315.0 | 1,068 | 2,568 | 2,069 | 1,859 | 874.0 |
| Grok 4.6 xhigh | 363.0 | 1,284 | 2,378 | 3,346 | 2,021 | 704.0 |
| MiMo V2.6 Pro | 879.0 | 1,239 | 5,454 | 5,592 | 4,304 | 2,299 |
| GLM 5.3 max | 984.0 | 0 | 7,293 | 7,421 | 5,531 | 2,868 |
| Kimi K3 max | 744.0 | 2,311 | 4,953 | 4,879 | 3,598 | 1,802 |
| MiMo V2.6 Flash | 984.0 | 1,574 | 5,223 | 5,749 | 4,309 | 2,342 |
| Gemini 4 Argon high | 423.0 | 1,976 | 2,712 | 4,309 | 2,523 | 218.0 |
| GPT-5.6 Sol max | 677.0 | 1,023 | 3,697 | 2,976 | 2,298 | 1,347 |
| GPT-6 Sol max | 492.0 | 1,139 | 3,307 | 2,915 | 2,119 | 1,088 |
| GLM 5.3 Flash max | 1,097 | 1,631 | 7,372 | 7,769 | 6,166 | 3,474 |
| Qwen 3.8 2.4T-A95B | 1,029 | 0 | 7,318 | 6,674 | 4,395 | 2,299 |
| DeepSeek V4.1 Flash max | 881.0 | 1,790 | 6,695 | 6,280 | 4,713 | 2,182 |
| GPT-6.1 Terra max | 525.0 | 1,139 | 3,154 | 2,430 | 1,957 | 1,587 |
| Ling 3.1 Flash | 707.0 | 0 | 6,622 | 7,337 | 4,920 | 2,295 |
| Qwen 3.8 27B xhigh | 619.0 | 1,190 | 4,737 | 5,314 | 3,784 | 1,718 |
All results
167 rows| # | Model | Effort | AA-Briefcase v1.1 | Subscription $ / task | API $ / task | Harness |
|---|---|---|---|---|---|---|
| 1 | Sonnet 5.5 | max | 1812 ±11 | $0.31 | $20 | Stirrup |
| 2 | Opus 5.5 | max | 1800 ±11 | $0.32 | $21 | Stirrup |
| 3 | Opus 5.5 | xhigh | 1760 ±11 | $0.19 | $12 | Stirrup |
| 4 | Sonnet 5.5 | xhigh | 1743 ±12 | $0.11 | $6.98 | Stirrup |
| 5 | Opus 5.5 | high | 1681 ±10 | $0.095 | $6.27 | Stirrup |
| 6 | Fable 5.1 | max | 1669 ±9 | $3.54 | $23 | Stirrup |
| 7 | Opus 5 | max | 1655 ±8 | Stirrup | ||
| 8 | Fable 5.1 | xhigh | 1650 ±8 | $2.77 | $18 | Stirrup |
| 9 | Grok 4.7 | xhigh | 1638 ±9 | $0.74 | $8.03 | Stirrup |
| 10 | Sonnet 5.5 | high | 1634 ±10 | $0.050 | $3.27 | Stirrup |
| 11 | Grok 4.7 | high | 1629 ±9 | $0.59 | $6.38 | Stirrup |
| 12 | Opus 5 | xhigh | 1629 ±8 | Stirrup | ||
| 13 | Opus 5.5 | medium | 1621 ±10 | $0.067 | $4.40 | Stirrup |
| 14 | Gemini 4 Carbon | high | 1609 ±10 | $8.16 | $8.16 | Stirrup |
| 15 | Muse Spark 1.3 | max | 1581 ±9 | $1.64 | $3.28 | Stirrup |
| 16 | Fable 5.1 | high | 1577 ±8 | $1.78 | $11 | Stirrup |
| 17 | Haiku 5.5 | max | 1575 ±10 | $0.014 | $0.67 | Stirrup |
| 18 | DeepSeek V4.1 Flash 1017 | max | 1570 ±10 | free | $0.68 | Stirrup |
| 19 | GPT-6 Astra | max | 1568 ±9 | $0.81 | $9.50 | Stirrup |
| 20 | Opus 5 | high | 1557 ±8 | Stirrup | ||
| 21 | GPT-6.1 Sol | max | 1555 ±10 | $0.27 | $2.31 | Stirrup |
| 22 | GPT-6 Astra | xhigh | 1543 ±9 | $0.56 | $6.56 | Stirrup |
| 23 | Grok 4.6 | xhigh | 1541 ±8 | Stirrup | ||
| 24 | Grok 4.6 | high | 1529 ±8 | Stirrup | ||
| 25 | Haiku 5.5 | xhigh | 1524 ±11 | $0.008 | $0.37 | Stirrup |
| 26 | Fable 5.1 | medium | 1524 ±8 | $1.34 | $8.58 | Stirrup |
| 27 | MiMo V2.6 Pro | 1511 ±10 | $0.32 | $0.38 | Stirrup | |
| 28 | GPT-6 Astra | high | 1504 ±8 | $0.41 | $4.76 | Stirrup |
| 29 | GLM 5.3 | max | 1504 ±9 | $0.59 | $5.65 | Stirrup |
| 30 | GPT-6.1 Sol | xhigh | 1502 ±11 | $0.12 | $1.04 | Stirrup |