Benchmarks · math
MGSM (Vals)
A multilingual benchmark for mathematical questions. Scores are the values published by VALS AI. Source benchmark version 1.
11 models · 11 results · updated 2026-01-09 · vals.ai ↗
Leaderboard
DeepSeek R1 xhigh
More from this benchmark
12 published measuresTime per task75 results · lower is better
Score vs any published measure11 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek R1 xhigh | 98.0 | 91.6 | 91.6 | 88.4 | 93.2 | 92.0 | 90.8 | 94.4 | 94.8 | 89.2 | 90.8 |
| DeepSeek V3 none | 97.6 | 90.8 | 91.6 | 91.6 | 93.6 | 93.2 | 91.6 | 92.0 | 96.0 | 87.2 | 88.4 |
| gpt-oss-120b high | 98.0 | 88.8 | 92.4 | 92.8 | 95.2 | 92.8 | 83.2 | 96.4 | 95.2 | 88.8 | 88.8 |
| Qwen 3 Max | 98.4 | 90.4 | 92.0 | 89.6 | 95.6 | 93.2 | 86.0 | 94.4 | 93.2 | 87.6 | 89.6 |
| Grok 3 none | 97.2 | 86.4 | 92.4 | 82.8 | 95.6 | 92.0 | 90.8 | 93.2 | 94.8 | 88.0 | 91.6 |
| GLM 4.5 xhigh | 96.0 | 90.0 | 90.8 | 89.6 | 94.4 | 92.8 | 84.8 | 92.8 | 92.4 | 86.4 | 89.2 |
| GLM 4.6 none | 99.2 | 89.6 | 91.2 | 74.4 | 94.8 | 92.0 | 82.8 | 92.4 | 91.2 | 88.4 | 91.2 |
| gpt-oss-20b high | 97.6 | 89.2 | 90.8 | 84.0 | 95.2 | 93.2 | 84.4 | 90.4 | 91.6 | 86.4 | 76.4 |
| GLM 4.7 none | 98.4 | 89.6 | 92.4 | 56.8 | 96.8 | 93.2 | 82.0 | 93.6 | 91.2 | 86.4 | 89.6 |
| Grok 4 Fast none | 95.2 | 84.8 | 89.2 | 78.4 | 92.8 | 91.2 | 83.6 | 91.6 | 92.0 | 85.2 | 84.0 |
| MiniMax M2.1 | 100.0 | 90.0 | 90.4 | 58.8 | 93.2 | 93.6 | 83.6 | 92.4 | 92.0 | 84.4 | 88.0 |
All results
11 rows| # | Model | Effort | MGSM (Vals) | Subscription $ / task | API $ / task | Time |
|---|---|---|---|---|---|---|
| 1 | DeepSeek R1 | xhigh | 92.3% | $0.004 | $0.004 | 10s |
| 2 | DeepSeek V3 | none | 92.1% | $0.000 | $0.000 | 16s |
| 3 | gpt-oss-120b | high | 92.0% | $0.001 | $0.001 | 5.8s |
| 4 | Qwen 3 Max | max | 91.8% | $0.003 | $0.003 | 11s |
| 5 | Grok 3 | none | 91.3% | $0.006 | $0.006 | 5.8s |
| 6 | GLM 4.5 | xhigh | 90.8% | $0.006 | $0.006 | 90s |
| 7 | GLM 4.6 | none | 89.7% | $0.004 | $0.004 | 34s |
| 8 | gpt-oss-20b | high | 89.0% | $0.000 | $0.000 | 5.8s |
| 9 | GLM 4.7 | none | 88.2% | $0.004 | $0.004 | 63s |
| 10 | Grok 4 Fast | none | 88.0% | $0.000 | $0.000 | 1.8s |
| 11 | MiniMax M2.1 | 87.9% | $0.001 | $0.001 | 14s |