- #63max of 2241342
- #76xhigh of 2241268
- #92high of 2241170
- #128medium of 224935
- #155low of 224741
- #163none of 224706
Leader: Sonnet 5.5 · 1823
- #35xhigh of 50-1.5%
Leader: Opus 5.5 · 15.5%
- #7max of 3335.8%
Leader: Gemini 3.5 Flash · 47.1%
agentic
AutomationBench (Zapier)
115 results- #43max of 11517.1%
- #61xhigh of 11512.9%
- #75high of 1159.1%
- #96medium of 1154.3%
- #106low of 1151.8%
Leader: Gemini 4 · 51.3%
agentic
AutomationBench-AA
219 results- #75max of 21950.2%
- #88xhigh of 21942.6%
- #102high of 21935.6%
- #124medium of 21923.1%
- #146low of 21911.7%
- #160none of 2195.0%
Leader: Gemini 4 · 77.5%
- #25max of 2761.5%
Leader: Sonnet 5.5 · 81.1%
agentic
EnterpriseOps-Gym-AA
50 results- #26max of 5040.8%
Leader: Fable 5 · 51.1%
agentic
Harvey's Legal Agent Benchmark
79 results- #55max of 791.3%
Leader: Muse Spark 1.2 · 25.4%
- #18max of 4540.3%
Leader: GPT-5.6 Sol · 56.2%
agentic
Legal Research Bench
78 results- #39max of 7836.5%
Leader: Muse Spark 1.3 · 55.3%
- #6max of 3460.4%
Leader: DeepSeek V4.1 Flash · 69.8%
- #37max of 6924.8%
Leader: Fable 5.1 · 49.2%
agentic
Terminal-Bench 2.1
18 results- #13max of 1875.7%
Leader: GPT-6 Astra · 87.4%
agentic
Terminal-Bench 4.0 (Vals)
47 results- #33max of 4711.6%
Leader: Opus 5.5 · 65.2%
agentic
Terminal-Bench Science (Vals)
40 results- #38max of 400.0%
Leader: GPT-6 Astra · 62.9%
- #40max of 67$4,095
Leader: GPT-6 Astra · $15,515
- #17xhigh of 175.2%
Leader: GPT-6 Astra · 42.2%
- #62max of 20631.1%
- #72xhigh of 20628.7%
- #82high of 20625.2%
- #106medium of 20617.7%
- #131low of 20612.8%
- #149none of 2069.7%
Leader: Qwen 3.8 Max · 51.3%
coding
Arena Image-to-WebDev
58 results- #37xhigh of 581517
Leader: Opus 5.5 · 1749
- #48xhigh of 1221519
Leader: Opus 5.5 · 1813
- #22max of 7744.5%
Leader: Sonnet 5.5 · 69.8%
coding
FrontierCode 1.1 Extended
130 results- #63max of 13055.1%
- #69xhigh of 13054.5%
- #88high of 13050.5%
- #107medium of 13041.2%
- #123low of 13029.0%
Leader: Opus 5.5 · 65.3%
coding
FrontierCode 1.1 Main
130 results- #72max of 13039.8%
- #74xhigh of 13038.9%
- #86high of 13035.9%
- #107medium of 13025.7%
- #122low of 13015.4%
Leader: Opus 5.5 · 54.6%
- #16max of 4461.8%
Leader: Gemini 4 · 100.0%
coding
ProgramBench v1 Almost Resolved
62 results- #31max of 629.0%
Leader: Opus 5.5 · 65.0%
coding
ProgramBench v1 Fully Resolved
62 results- #56max of 620.0%
Leader: Opus 5.5 · 18.5%
coding
ProgramBench v1 Raw Pass Rate
62 results- #27max of 6268.3%
Leader: Opus 5.5 · 87.0%
- #73max of 22653.6%
- #90high of 22651.6%
- #101xhigh of 22650.5%
- #131medium of 22646.8%
- #135low of 22646.1%
- #165none of 22640.4%
Leader: Opus 5.5 · 66.9%
coding
SWE-bench Verified (Vals)
85 results- #9max of 8593.0%
Leader: Opus 5 · 97.0%
coding
Terminal-Bench 2.1 (Vals)
74 results- #11max of 7479.0%
Leader: Opus 5.5 · 87.6%
coding
Terminal-Bench Science-AA
46 results- #36max of 463.3%
Leader: GPT-6 Astra · 63.3%
coding
Terminal-Bench v2.1-AA
238 results- #41max of 23880.9%
- #54xhigh of 23877.9%
- #73high of 23869.7%
- #113medium of 23853.2%
- #134low of 23843.4%
- #144none of 23839.0%
Leader: Fable 5.1 · 91.4%
coding
Terminal-Bench v4-AA
216 results- #83max of 21611.6%
- #103xhigh of 2163.5%
- #108high of 2162.5%
- #134none of 2161.0%
- #138medium of 2160.5%
- #161low of 2160.0%
Leader: Sonnet 5.5 · 63.6%
coding
Vibe Code Bench 1-100
22 results- #5max of 2222.6%
Leader: Opus 5.5 · 30.4%
coding
Vibe Code Bench v1.1
105 results- #31max of 10577.1%
Leader: Sonnet 5.5 · 92.4%
- #42max of 12760.9%
Leader: GPT-6 Astra Pro · 93.6%
composite
AA Intelligence Index v4.3.2
610 results- #79max of 61037.3
- #87xhigh of 61034.6
- #108high of 61032.1
- #161medium of 61025.0
- #212low of 61021.0
- #277none of 61015.5
Leader: Opus 5.5 · 57.6
- #64xhigh of 2201452
Leader: Gemini 4 · 1525
composite
Arena Text — Multi-Turn
220 results- #67xhigh of 2201457
Leader: Gemini 4 · 1553
- #25max of 4751.7%
Leader: Gemini 4 · 68.9%
composite
Vals Multimodal Index v1.2
33 results- #7max of 3369.1%
Leader: Fable 5 · 74.2%
cost
AA Intelligence Index Cost per Task
177 results- #4low of 177$0.01
- #5none of 177$0.01
- #10medium of 177$0.02
- #21high of 177$0.04
- #35xhigh of 177$0.09
- #53max of 177$0.18
Leader: GPT-6 Luna · $0.00
cost
Cost to Run AA Intelligence Index
177 results- #5none of 177$22.97
- #7low of 177$27.02
- #15medium of 177$44.32
- #25high of 177$108
- #40xhigh of 177$179
- #52max of 177$320
Leader: GPT-6 Luna · $10.63
- #27high of 341291
Leader: Opus 5.5 · 2079
creative
VoxelBench (text)
55 results- #13max of 551856
Leader: GPT-6 Astra · 2732
- #40max of 11764.2%
Leader: Opus 5 · 73.2%
economics
Excel Modeling Benchmark
74 results- #17max of 7467.1%
Leader: Fable 5.1 · 76.7%
economics
Finance Agent v2
78 results- #22max of 7855.0%
Leader: Gemini 4 · 65.4%
- #39max of 21024.0%
- #41xhigh of 21023.8%
- #49high of 21022.2%
- #94medium of 21015.2%
- #98low of 21014.2%
- #150none of 2106.0%
Leader: Muse Wonder Full (Max) · 32.6%
- #56max of 2881459
- #72xhigh of 2881399
- #90high of 2881330
- #125medium of 2881129
- #162low of 288994
- #173none of 288919
Leader: Opus 5.5 · 1866
economics
MortgageTax (Vals)
84 results- #26max of 8467.3%
Leader: Opus 5 · 72.1%
- #7max of 12576.2%
Leader: Muse Spark 1.2 · 80.4%
knowledge
AA-Omniscience Accuracy
526 results- #104max of 52642.7%
- #105xhigh of 52642.4%
- #107high of 52641.8%
- #118medium of 52640.7%
- #124low of 52639.6%
- #191none of 52628.6%
Leader: Fable 5.1 · 67.2%
knowledge
AA-Omniscience Non-hallucination
526 results- #238none of 52625.0%
- #401low of 52610.1%
- #415medium of 5269.1%
- #456high of 5267.6%
- #458xhigh of 5267.5%
- #460max of 5267.4%
Leader: MiniCPM5-1B (Non-reasoning) · 99.1%
- #51max of 53991.1%
- #74xhigh of 53989.5%
- #80high of 53989.2%
- #118medium of 53985.9%
- #156low of 53983.5%
- #336none of 53964.5%
Leader: GPT-6 Astra · 96.3%
knowledge
GPQA Diamond (Vals)
122 results- #18max of 12291.7%
Leader: Gemini 3.1 Pro · 95.5%
- #48max of 9442.4%
Leader: Opus 5 · 63.6%
- #34max of 9684.4%
Leader: Opus 5.5 · 91.4%
- #49max of 12286.0%
Leader: Fable 5.1 · 92.4%
- #25max of 8185.0%
Leader: Fable 5.1 · 90.6%
- #65max of 26578.6%
- #66xhigh of 26578.6%
- #77high of 26577.6%
- #91medium of 26575.8%
- #113low of 26574.0%
- #199none of 26560.3%
Leader: Opus 5.5 · 87.7%
knowledge
Public Benefits Bench v1.1
50 results- #31max of 5061.2%
Leader: Opus 5 · 76.9%
- #19max of 52383.7%
- #51xhigh of 52381.7%
- #74high of 52380.3%
- #154medium of 52375.0%
- #213low of 52370.0%
- #330none of 52342.7%
Leader: Kimi K3 · 88.7%
long-context
Arena Document
43 results- #23xhigh of 431457
Leader: Opus 5 · 1516
- #33max of 9419.4%
- #44xhigh of 9416.1%
- #58high of 9412.2%
- #70medium of 948.3%
- #75low of 946.1%
Leader: Sonnet 5.5 · 75.0%
math
FrontierMath Tiers 1–3 (v2)
115 results- #14max of 11582.1%
- #57low of 11541.4%
- #58none of 11539.6%
Leader: GPT-6.1 Sol · 93.7%
- #24max of 5260.0%
Leader: Sonnet 5.5 · 100.0%
- #47xhigh of 1161258
Leader: Fable 5 · 1308
- #9max of 851.5%
Leader: Opus 5.5 · 29.1%
- #64max of 21590.7%
- #80xhigh of 21587.7%
- #110high of 21578.0%
- #145medium of 21556.5%
- #166low of 21539.5%
- #212none of 2153.3%
Leader: Fable 5 · 98.5%
- #80max of 21659.6%
- #98xhigh of 21647.6%
- #110high of 21633.3%
- #145medium of 2167.4%
- #153low of 2165.1%
- #192none of 2160.8%
Leader: GPT-6 Astra · 95.0%
reasoning
Blueprint-Bench 2
28 results- #25max of 2822.6%
Leader: Gemini 4 · 54.4%
- #57max of 52920.6%
- #58xhigh of 52920.6%
- #83high of 52916.6%
- #154medium of 5294.9%
- #186low of 5292.6%
- #277none of 5290.3%
Leader: Muse Wonder Full (Max) · 32.9%
reasoning
Furniture Assembly
31 results- #14max of 3142.5%
Leader: Opus 5.5 · 83.3%
reasoning
Humanity's Sixth Sense
25 results- #24max of 2521.6%
Leader: GPT-6 Astra · 53.6%
- #42max of 13584.0%
Leader: Fable 5 · 88.6%
reasoning
MysteryMechanism
27 results- #26max of 2714.4%
Leader: GPT-6 Astra · 53.2%
reasoning
Roboflow Visual Reasoning
102 results- #49high of 10265.6%
- #62low of 10260.5%
Leader: GPT-6 Astra · 91.2%
- #45max of 8044.2%
Leader: Opus 4.7 · 56.1%
reasoning
SimpleQA Verified
47 results- #22max of 4739.1%
Leader: Gemini 3.1 Pro · 77.5%
reliability
Tool-call error rate
171 results- #65max of 1710.9%
Leader: GPT-5.4 Pro · 0.0%
- #129max of 18198.6%
Leader: GPT-5 Pro · 100.0%
- #11max of 4573.6%
Leader: Wonder · 80.4%
- #24max of 321.5%
Leader: GPT-6 Astra · 56.9%
speed
Latency (Time to First Token)
351 results- #26none of 3510.7s
- #119low of 3511.6s
- #176medium of 3512.4s
- #297high of 35129s
- #311xhigh of 35145s
- #334max of 351117s
Leader: Gemini 2.5 Flash-Lite · 0.3s
speed
Output Speed (tokens/s)
351 results- #120max of 351122.2
- #125xhigh of 351118.7
- #136high of 351111.9
- #137none of 351111.8
- #142medium of 351110.2
- #148low of 351106.9
Leader: Celeris-1 · 1522.5
- #116max of 1771.8s
Leader: Qwen 2.5 7B · 0.2s
- #33max of 17698.0
Leader: Muse Spark 1.1 · 204.3