Benchmarks · Vision · Reasoning & puzzles
ShadowBench
Hand shadows: shown a plate from a nineteenth-century book, the model poses two 3D hands between a lamp and a screen, one tool call at a time, until their shadow matches the animal. Poses a real hand could not make are refused. Scored 0 to 100 on overlap and outline against the target, averaged over 20 plates, with token cost per episode. No model has solved a plate. From Perpetual Research. Reference scores, not models: Featureless blob 56.9, Blind search (CMA-ES) 36.2, Hands at rest 12.1.
6 models · 18 results · updated 2026-11-05 · perpetualresearch.si ↗
Leaderboard
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data.
More from this benchmark
6 published measuresAgent steps per task18 results · lower is better
Time per task18 results · lower is better
Score vs any published measure4 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||
|---|---|---|---|---|
| Opus 5.5 max | 0.815 | 0.285 | 20.0 | 68.3 |
| Sonnet 5.5 max | 0.815 | 0.278 | 20.0 | 60.0 |
| GPT-6.1 Terra max | 0.772 | 0.245 | 20.0 | 61.4 |
| GPT-6.1 Sol max | 0.766 | 0.242 | 20.0 | 55.5 |
| Haiku 5.5 max | 0.703 | 0.21 | 20.0 | 60.0 |
| GPT-6 Luna max | 0.609 | 0.148 | 20.0 | 49.3 |
All results
18 rows| # | Model | Effort | ShadowBench | Subscription $ / task | API $ / task | Agent steps | Time |
|---|---|---|---|---|---|---|---|
| 1 | Opus 5.5 | max | 48.8 ±2.8 | $0.11 | $7.13 | 97 | 54 min |
| 2 | Sonnet 5.5 | max | 48.8 ±2.3 | $0.079 | $5.23 | 97 | 60 min |
| 3 | GPT-6.1 Terra | max | 46.4 ±2.7 | $0.017 | $0.44 | 99 | 22 min |
| 4 | GPT-6.1 Sol | max | 44.7 ±2.8 | $0.030 | $0.88 | 100 | 40 min |
| 5 | GPT-6.1 Terra | high | 44.4 ±3.1 | $0.008 | $0.21 | 98 | 23 min |
| 6 | Opus 5.5 | high | 43.6 ±2.9 | $0.019 | $1.27 | 93 | 23 min |
| 7 | GPT-6.1 Terra | medium | 43.4 ±1.8 | $0.007 | $0.18 | 97 | 25 min |
| 8 | GPT-6.1 Sol | high | 41.9 ±2.7 | $0.013 | $0.39 | 100 | 35 min |
| 9 | GPT-6.1 Sol | medium | 41.4 ±3.1 | $0.011 | $0.32 | 99 | 39 min |
| 10 | Haiku 5.5 | max | 41.3 ±3.7 | $0.007 | $0.34 | 95 | 54 min |
| 11 | Sonnet 5.5 | high | 40.6 ±2.6 | $0.006 | $0.41 | 65 | 13 min |
| 12 | Opus 5.5 | medium | 39.8 ±3.5 | $0.013 | $0.85 | 85 | 22 min |
| 13 | Haiku 5.5 | high | 35.6 ±3.1 | $0.001 | $0.036 | 64 | 12 min |
| 14 | GPT-6 Luna | max | 35.1 ±3.4 | $0.007 | $0.098 | 89 | 24 min |
| 15 | GPT-6 Luna | high | 35.0 ±2.3 | $0.004 | $0.059 | 90 | 20 min |
| 16 | GPT-6 Luna | medium | 31.6 ±3.0 | $0.003 | $0.041 | 90 | 21 min |
| 17 | Haiku 5.5 | medium | 28.5 ±3.2 | $0.000 | $0.012 | 35 | 7.4 min |
| 18 | Sonnet 5.5 | medium | 17.3 ±5.3 | $0.000 | $0.030 | 8 | 54s |