Benchmarks · agentic
Toolathlon, original release (HKUST NLP, archived board)
An agent completes 108 real-world tool-use tasks across MCP servers and local toolkits, graded by executing the resulting state against each task's ground truth — the original task definitions, before the June 2026 Verified repair. Maintained by HKUST NLP. Score: Pass@1 in percent as published on the archived pre-Verified board.
33 models · 34 results · data as of 2026-10-29 · toolathlon.xyz ↗
Leaderboard
Gemini 3.5 Flash xhigh
All results
34 rows| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Gemini 3.5 Flash | xhigh | 56.5% |
| 2 | Opus 4.7 | max | 52.8% |
| 3 | DeepSeek V4 Pro | max | 52.8% |
| 4 | Gemini 3 Flash | high | 49.4% |
| 5 | Gemini 3.1 Pro | high | 48.8% |
| 6 | DeepSeek V4 Flash | max | 48.2% |
| 7 | Sonnet 4.6 | max | 44.8% |
| 8 | GPT-5.2 | xhigh | 43.8% |
| 9 | Opus 4.5 | high | 43.5% |
| 10 | GPT-5.2 | high | 41.7% |
| 11 | MiniMax M2.1 | 40.7% | |
| 12 | GLM 5 | high | 39.2% |
| 13 | Sonnet 4.5 | high | 38.9% |
| 14 | GPT-5 | high | 37.7% |
| 15 | Qwen 3.5 Plus | none | 37.7% |
| 16 | GPT-5.1 | high | 37.0% |
| 17 | Gemini 3 Pro | high | 36.4% |
| 18 | DeepSeek V3.2 | high | 35.2% |
| 19 | Sonnet 4 | high | 29.9% |
| 20 | Kimi K2.5 | max | 27.8% |
| 21 | Grok 4 | high | 27.5% |
| 22 | Haiku 4.5 | xhigh | 26.2% |
| 23 | GLM 4.7 | none | 23.8% |
| 24 | GLM 4.6 | none | 18.8% |
| 25 | Grok 4 Fast | none | 18.5% |
| 26 | Kimi K2 | medium | 17.6% |
| 27 | o3 | high | 17.0% |
| 28 | o4 mini | high | 14.8% |
| 29 | GPT-5 Mini | high | 14.5% |
| 30 | Qwen 3 Coder | medium | 14.5% |