Benchmarks · knowledge
MedQA (Vals)
Evaluating language model bias in medical questions. Scores are the values published by VALS AI. Source benchmark version 1.
15 models · 15 results · updated 2026-04-16 · vals.ai ↗
Leaderboard
GLM 4.7 none
More from this benchmark
7 published measuresTime per task94 results · lower is better
Score vs any published measure6 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||||
|---|---|---|---|---|---|---|
| GLM 4.7 none | 94.0 | 93.5 | 94.3 | 94.0 | 94.1 | 92.5 |
| MiniMax M2.5 high | 93.2 | 92.3 | 92.1 | 92.6 | 92.3 | 92.6 |
| GLM 4.6 none | 92.2 | 92.0 | 91.8 | 92.8 | 92.2 | 92.3 |
| Sonnet 4.6 max | 92.2 | 91.9 | 92.8 | 91.6 | 92.0 | 91.9 |
| gpt-oss-120b high | 91.9 | 91.2 | 90.8 | 91.4 | 91.3 | 91.6 |
| MiniMax M2.1 | 92.3 | 91.1 | 91.2 | 90.4 | 90.8 | 91.3 |
| DeepSeek R1 xhigh | 91.5 | 90.5 | 91.2 | 90.5 | 91.0 | 90.1 |
| GLM 4.5 xhigh | 91.4 | 89.8 | 90.0 | 89.9 | 89.8 | 89.0 |
| Qwen 3 Max | 88.6 | 86.9 | 87.3 | 86.9 | 87.5 | 87.0 |
| Grok 3 none | 85.3 | 83.2 | 84.3 | 83.7 | 83.8 | 82.8 |
| gpt-oss-20b high | 83.9 | 83.1 | 83.5 | 82.7 | 81.7 | 82.4 |
| DeepSeek V3 none | 82.3 | 80.5 | 80.9 | 80.5 | 81.3 | 79.8 |
| Grok 4 Fast none | 77.1 | 75.3 | 74.8 | 75.0 | 75.4 | 74.6 |
| Jamba 1.5 Large none | 69.7 | 68.2 | 67.6 | 67.7 | 68.5 | 67.2 |
| Jamba 1.5 Mini none | 56.5 | 55.0 | 54.4 | 55.3 | 55.9 | 54.0 |
All results
15 rows| # | Model | Effort | MedQA (Vals) | Subscription $ / task | API $ / task | Time |
|---|---|---|---|---|---|---|
| 1 | GLM 4.7 | none | 93.7% | $0.005 | $0.005 | 66s |
| 2 | MiniMax M2.5 | high | 92.5% | $0.003 | $0.003 | 34s |
| 3 | GLM 4.6 | none | 92.2% | $0.004 | $0.004 | 29s |
| 4 | Sonnet 4.6 | max | 92.1% | $0.035 | $0.035 | 35s |
| 5 | gpt-oss-120b | high | 91.4% | $0.001 | $0.001 | 12s |
| 6 | MiniMax M2.1 | 91.2% | $0.002 | $0.002 | 36s | |
| 7 | DeepSeek R1 | xhigh | 90.8% | $0.006 | $0.006 | 42s |
| 8 | GLM 4.5 | xhigh | 90.0% | $0.003 | $0.003 | 60s |
| 9 | Qwen 3 Max | max | 87.4% | $0.003 | $0.003 | |
| 10 | Grok 3 | none | 83.8% | $0.006 | $0.006 | 7.5s |
| 11 | gpt-oss-20b | high | 82.9% | $0.000 | $0.000 | 25s |
| 12 | DeepSeek V3 | none | 80.9% | $0.001 | $0.001 | 6.8s |
| 13 | Grok 4 Fast | none | 75.4% | $0.000 | $0.000 | 2.4s |
| 14 | Jamba 1.5 Large | none | 68.1% | $0.002 | $0.002 | 6.0s |
| 15 | Jamba 1.5 Mini | none | 55.2% | $0.000 | $0.000 | 1.1s |