Benchmarks · agentic

Agent Arena · Shell reliability

Agent Arena signal: running shell commands with the fewest failures. A model's contribution to real agentic sessions, in percentage points.

All results

46 rows

#	Model	Effort	Score
1	Sonnet 5	high	15.2% ±0.5
2	Opus 5.5	max	15.0% ±0.5
3	Sonnet 5.5	max	12.7% ±0.7
4	GPT-6 Sol	max	11.5% ±0.6
5	GPT-5.6 Sol	xhigh	11.3% ±0.7
6	Fable 5	high	10.7% ±0.9
7	Opus 5	max	10.4% ±0.6
8	Gemini 3.8 Flash	high	10.1% ±0.4
9	Fable 5.1	max	10.1% ±1.0
10	GPT-6 Luna	max	10.0% ±0.3
11	Grok 4.6	xhigh	9.9% ±1.0
12	Grok 4.7	xhigh	9.5% ±0.6
13	GPT-6 Astra	max	8.6% ±1.2
14	Haiku 5.5	high	7.5% ±0.6
15	Gemini 3.7 Flash	high	6.2% ±0.9
16	GPT-6.1 Sol	max	5.6% ±0.7
17	Qwen 3.7 Max	max	3.6% ±2.6
18	Inkling Small	max	3.5% ±2.9
19	MiMo V2.6 Pro	high	3.1% ±0.6
20	Muse Spark 1.3	max	2.0% ±0.8
21	DeepSeek V4 Pro	max	1.9% ±1.3
22	Qwen 3.7 Plus	high	1.7% ±2.8
23	Gemini 3.1 Pro	high	1.5% ±3.1
24	GPT-6 Luna	low	1.4% ±0.8
25	GPT-5.6 Terra	xhigh	0.8% ±0.8
26	Inkling	max	0.8% ±3.3
27	Mistral Large 4	high	0.1% ±2.0
28	Nemotron 3 Ultra 550B A55B	max	-16.7% ±12.6
29	DeepSeek V4.1 Flash	max	-0.8% ±0.2
30	Gemini 4 Carbon	high	-5.4% ±0.9