Benchmarks · reliability
Tool-call error rate
Volume-weighted mean tool-call error rate across OpenRouter providers (3d).
175 models · 175 results · data as of 2026-10-29 · openrouter.ai ↗
Leaderboard
GPT-5 Pro high
All results
175 rows| # | Model | Effort | Tool-call error rate |
|---|---|---|---|
| 1 | GPT-5 Pro | high | 0.0% |
| 2 | GPT-5.4 Pro | xhigh | 0.0% |
| 3 | GPT-5.5 Pro | xhigh | 0.0% |
| 4 | o3 pro | high | 0.0% |
| 5 | GPT-6.1 Sol Pro | 0.0% | |
| 6 | GPT-6 Sol Pro | max | 0.1% |
| 7 | Qwen 3 30B-A3B | none | 0.1% |
| 8 | Kimi K3.1 | max | 0.1% |
| 9 | Grok 4.20 | xhigh | 0.1% |
| 10 | GPT-6 Astra | max | 0.1% |
| 11 | GPT-6 Astra Pro | max | 0.1% |
| 12 | Grok 4.3 | xhigh | 0.1% |
| 13 | Kimi K3 | max | 0.1% |
| 14 | Hy4 | high | 0.2% |
| 15 | GPT-6.1 Sol | max | 0.2% |
| 16 | GPT-5.6 Terra Pro | 0.2% | |
| 17 | Grok 4.7 | max | 0.3% |
| 18 | Grok 4.6 | max | 0.3% |
| 19 | GPT-5.6 Sol Pro | max | 0.3% |
| 20 | GPT-6.1 Terra Pro | 0.3% | |
| 21 | o3 | high | 0.3% |
| 22 | Grok 4.5 | max | 0.3% |
| 23 | GPT-6 Sol | max | 0.3% |
| 24 | Qwen 3.5 Plus | none | 0.3% |
| 25 | GPT-5.5 | xhigh | 0.4% |
| 26 | GPT-5.3 Codex | xhigh | 0.4% |
| 27 | Muse Spark 1.1 | max | 0.4% |
| 28 | Opus 5 | max | 0.5% |
| 29 | Opus 4.6 | max | 0.5% |
| 30 | GPT-6.1 Terra | max | 0.5% |