Benchmarks · agentic

Agent Arena · Tool hallucination

Agent Arena signal: calling tools that do not exist (lower is better). A model's contribution to real agentic sessions, in percentage points.

All results

46 rows

#	Model	Effort	Score
1	Nemotron 3 Ultra 550B A55B	max	-0.6% ±1.1
2	Haiku 5.5	high	-1.0% ±0.6
3	Gemini 3.1 Pro	high	-0.1% ±0.6
4	Hy3	xhigh	-4.5% ±2.4
5	Inkling Small	max	-2.6% ±1.5
6	Inkling	max	-1.9% ±1.4
7	MiMo V2.5 Pro	xhigh	-1.2% ±0.4
8	MiMo V2.6 Flash	high	-3.2% ±0.4
9	MiMo V2.6 Pro	high	-0.2% ±0.3
10	MiniMax M3	max	-1.7% ±2.4
11	Mistral Large 4	high	-19.3% ±2.9
12	Mistral Medium 3.5	high	-3.7% ±1.4
13	Qwen 3.7 Max	max	-0.4% ±0.4
14	Qwen 3.7 Plus	high	-3.2% ±3.3
15	Qwen 3.8 27B	max	-0.1% ±0.2
16	Qwen 3.8 Flash	max	-0.8% ±0.2
17	Qwen 3.8 Max	max	-0.8% ±0.2
18	Hy4	high	0.1% ±0.2
19	Muse Spark 1.2	xhigh	0.3% ±0.2
20	Muse Spark 1.1	max	0.3% ±0.1
21	Gemini 4 Argon	high	0.3% ±0.1
22	Gemini 3.8 Flash	high	0.3% ±0.1
23	Gemini 3.7 Flash	high	0.4% ±0.1
24	Sonnet 5	high	0.4% ±0.1
25	Opus 5.5	max	0.5% ±0.2
26	Gemini 4 Carbon	high	0.5% ±0.2
27	Fable 5	high	0.5% ±0.2
28	DeepSeek V4.1 Flash	max	0.6% ±0.1
29	Grok 4.6	xhigh	0.6% ±0.1
30	DeepSeek V4 Pro	max	0.6% ±0.1