Benchmarks · agentic
FrontierPhysics
End-to-end physics research by agents: 56 tasks across nine domains (theory, computation and experiment), written by 42 physicists from 26 institutions, 3 trials each. A pass needs every deterministic verifier test and every blocker rubric. Value is the overall pass rate averaged over the trials. Cost is per rollout, one attempt at one task.
22 models · 26 results · updated 2026-11-04 · benchflow.ai ↗
Leaderboard
Opus 5.5 max
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data. 12 more models under All models in settings.
More from this benchmark
9 published measuresAgent steps per task26 results · lower is better
13 more models under All models in settings.
Time per task26 results · lower is better
Output tokens per task26 results · lower is better
Score vs any published measure6 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||||
|---|---|---|---|---|---|---|
| Opus 5.5 max | 45.8 | 62.5 | 44.6 | 26.0 | 46.0 | 25.0 |
| GPT-6 Astra max | 39.3 | 51.8 | 30.4 | 21.8 | 39.0 | 17.0 |
| GPT-6.1 Sol max | 33.3 | 50.0 | 30.4 | 18.4 | 33.0 | 17.0 |
| GPT-6 Sol max | 27.4 | 44.6 | 28.6 | 12.5 | 24.0 | 16.0 |
| GPT-6.1 Terra max | 28.6 | 39.3 | 21.4 | 12.6 | 24.0 | 12.0 |
| Fable 5.1 max | 31.0 | 44.6 | 21.4 | 10.2 | 19.0 | 12.0 |
| Opus 5 max | 20.2 | 30.4 | 17.9 | 8.6 | 16.0 | 10.0 |
| Gemini 4 Argon high | 31.5 | 46.4 | 17.9 | 6.5 | 14.0 | 10.0 |
| GPT-5.6 Sol max | 19.6 | 28.6 | 12.5 | 6.2 | 12.0 | 7 |
| Kimi K3.1 max | 15.5 | 30.4 | 12.5 | 6.1 | 11.0 | 7 |
| Haiku 5.5 max | 15.5 | 30.4 | 10.7 | 3.6 | 7 | 6 |
| GLM 5.3 max | 7.1 | 14.3 | 8.9 | 3.6 | 7 | 5 |
| Grok 4.7 xhigh | 16.7 | 35.7 | 10.7 | 3.8 | 7 | 6 |
| DeepSeek V4.1 Flash max | 10.7 | 14.3 | 5.4 | 3.3 | 6 | 3 |
| GPT-5.6 Luna max | 12.5 | 21.4 | 3.6 | 2.9 | 6 | 2 |
| GPT-5.6 Terra max | 16.1 | 25.0 | 7.1 | 2.9 | 6 | 4 |
| Grok 4.6 max | 10.7 | 21.4 | 3.6 | 2.2 | 4 | 2 |
| Kimi K3 max | 11.3 | 21.4 | 5.4 | 1.8 | 4 | 3 |
| MiMo V2.6 Pro max | 10.1 | 19.6 | 3.6 | 1 | 2 | 2 |
| DeepSeek V4 Pro max | 3 | 5.4 | 1.8 | 0.3 | 1 | 1 |
| Gemini 3.8 Flash high | 7.7 | 14.3 | 0 | 0 | 0 | 0 |
| Mistral Large 4 max | 5.4 | 12.5 | 0 | 0 | 0 | 0 |
All results
26 rows| # | Model | Effort | FrontierPhysics | Subscription $ / task | API $ / task | Agent steps | Time | Output tokens | Harness |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Opus 5.5 | max | 27.4% ±6.7 | $0.41 | $27 | 219 | 2.4 h | 382k | Claude Code |
| 2 | GPT-6 Astra | max | 23.2% ±6.4 | $1.15 | $27 | 194 | 1.7 h | 180k | Codex |
| 3 | GPT-6 Astra | xhigh | 19.6% ±6.0 | $0.82 | $19 | 78 | 1.2 h | 111k | Codex |
| 4 | GPT-6.1 Sol | max | 19.6% ±6.0 | $0.31 | $5.33 | 281 | 1.8 h | 206k | Codex |
| 5 | GPT-6 Sol | max | 14.3% ±5.3 | $0.20 | $6.38 | 179 | 1.1 h | 112k | Codex |
| 6 | GPT-6.1 Terra | max | 14.3% ±5.3 | $0.17 | $3.46 | 203 | 1.1 h | 123k | Codex |
| 7 | Fable 5.1 | max | 11.3% ±4.8 | $5.09 | $33 | 132 | 1.9 h | 266k | Claude Code |
| 8 | Opus 5 | max | 9.5% ±4.4 | $0.98 | $57 | 163 | 2.4 h | 305k | Claude Code |
| 9 | Gemini 4 Argon | high | 8.3% ±4.2 | $6.32 | $8.87 | 132 | 1.6 h | 311k | Antigravity |
| 10 | GPT-5.6 Sol | max | 7.1% ±3.9 | $0.29 | $10 | 117 | 58 min | 95k | Codex |
| 11 | Kimi K3.1 | max | 6.5% ±3.7 | $9.60 | $9.60 | 237 | 1.5 h | 166k | Claude Code |
| 12 | Haiku 5.5 | max | 4.2% ±3.0 | $0.14 | $6.66 | 224 | 2.5 h | 562k | Claude Code |
| 13 | GLM 5.3 | max | 4.2% ±3.0 | $1.33 | $13 | 349 | 3.2 h | 578k | Claude Code |
| 14 | GPT-5.6 Sol | xhigh | 4.2% ±3.0 | $0.077 | $2.79 | 44 | 25 min | 37k | Codex |
| 15 | Grok 4.7 | xhigh | 4.2% ±3.0 | $1.13 | $12 | 239 | 2.7 h | 172k | Grok Build |
| 16 | DeepSeek V4.1 Flash | max | 3.6% ±2.8 | $0.048 | $0.67 | 197 | 1.2 h | 269k | Claude Code |
| 17 | GPT-5.6 Luna | max | 3.6% ±2.8 | $0.032 | $0.68 | 150 | 1.0 h | 108k | Codex |
| 18 | GPT-5.6 Terra | max | 3.6% ±2.8 | $0.16 | $5.80 | 139 | 1.1 h | 100k | Codex |
| 19 | Grok 4.6 | max | 2.4% ±2.3 | $0.96 | $9.36 | 141 | 1.6 h | 79k | Grok Build |
| 20 | Kimi K3 | max | 2.4% ±2.3 | $0.85 | $9.33 | 160 | 2.0 h | 209k | Claude Code |
| 21 | GPT-5.6 Luna | xhigh | 1.8% ±1.9 | $0.019 | $0.41 | 98 | 45 min | 74k | Codex |
| 22 | GPT-5.6 Terra | xhigh | 1.2% ±1.4 | $0.061 | $2.22 | 48 | 35 min | 60k | Codex |
| 23 | MiMo V2.6 Pro | max | 1.2% ±1.4 | $0.56 | $0.66 | 202 | 3.0 h | 283k | Claude Code |
| 24 | DeepSeek V4 Pro | max | 0.6% ±0.9 | $1.30 | $7.82 | 97 | 2.1 h | 234k | Claude Code |
| 25 | Gemini 3.8 Flash | high | 0.0% ±0.0 | $0.15 | $3.69 | 197 | 27 min | 197k | Antigravity |
| 26 | Mistral Large 4 | max | 0.0% ±0.0 | $0.51 | $11 | 242 | 4.8 h | 880k | Claude Code |