Benchmarks
312 benchmarks, each with its own leaderboard and everything its publisher releases beyond the score. Where a benchmark measures cost per task, Priors adjusts it for subscription plans, so you see what a model costs to actually use.
/
312 benchmarks
Coding
49- Terminal-Bench v4-AAAA's 66-task Terminal-Bench v4.0, three repeats, average pass@1. Distinct from the upstream leaderboard and v2.1.150 models · in 12 mixes · quality 100 · artificialanalysis.ai
- ProgramBench v1 Fully Resolved · 2 categoriesShare of public ProgramBench v1 tasks where the implementation passes every hidden behavioral test.62 models · in 2 mixes · quality 99 · vals.ai
- FrontierCode 2.0-AAAA's run of Cognition's FrontierCode 2.0 (132 private tasks in real open-source repositories, three attempts each): would a maintainer merge the change? An attempt failing any blocking check scores 0. Separate from Cognition's own leaderboard, which uses different harnesses.39 models · in 5 mixes · quality 83 · artificialanalysis.ai
- DeepSWE v1.1Pass@1 on 113 original long-horizon software engineering tasks across 91 repos and 5 languages, run with mini-swe-agent in isolated environments (v1.1 fixes dependency drift and flaky tasks).28 models · in 5 mixes · quality 76 · deepswe.datacurve.ai
- Arena WebDev · 3 categoriesHuman preference between agentic front-end web development outputs; Bradley-Terry rating.113 models · in 4 mixes · quality 62 · arena.ai
- FrontierSWE v2Cognition's ultra-long-horizon software engineering benchmark: 34 tasks graded 0-100 on implementation, performance and research; mean@5 with a 20-hour budget per task in the Proximus harness at max reasoning effort.22 models · in 2 mixes · quality 67 · frontierswe.com
- FrontierCode 2.0Mergeability of coding-agent changes on private maintainer-authored tasks, graded by unit tests, rubrics and verifiers (Cognition).23 models · in 1 mix · quality 71 · cognition.com
- SciCodeSciCode: implementing scientific research code from paper descriptions; sub-problem accuracy.158 models · in 2 mixes · quality 60 · artificialanalysis.ai
Math
26- FrontierMath Tier 4 (v2) · 1 categoriesEpoch AI's FrontierMath Tier 4, version 2: expert-written mathematics problems evaluated with Python tools. Accuracy on the private set, not the public sample questions. Data: Epoch AI, CC BY.68 models · in 7 mixes · quality 90 · epoch.ai
- ProofBench v1.1Automated theorem proving benchmark53 models · in 1 mix · quality 60 · vals.ai
- MathArenaUses MathArena's website-wide IRT expected-performance estimate. Score: website-wide IRT expected performance. Collected by Jake Boggs' benchmark dashboard.84 models · in 1 mix · quality 56 · matharena.ai
- Riemann-BenchUses Surge AI's first-party extreme-math leaderboard score. Score: score. Collected by Jake Boggs' benchmark dashboard.41 models · quality 60 · surgehq.ai
- OTIS Mock AIME 2024-2025The OTIS Mock AIME problems (2024 and 2025), run by Epoch AI. Accuracy over repeated samples. Data: Epoch AI, CC BY.163 models · quality 59 · epoch.ai
- FrontierMath ErdősEpoch AI's 68 open Erdős problems, formulated in Lean. Models must supply complete formal proofs or disproofs. Published solved fraction; distinct from FrontierMath difficulty tiers. Data: Epoch AI, CC BY.13 models · quality 59 · epoch.ai
- Apex 2025 (MathArena)Twelve final-answer problems from 2025 competitions that MathArena selected because strong models of the time failed them; the selection is biased against the four models used to choose it (Grok 4, GPT-5, Gemini 2.5 Pro, GLM 4.5). Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).41 models · quality 50 · matharena.ai
- MathArena Apex · 1 categoriesMathArena Apex: 12 final-answer competition problems that earlier frontier models (Grok 4, GPT-5, Gemini 2.5 Pro, GLM 4.5) failed, so it is biased against them. Accuracy over repeated attempts.41 models · quality 50 · matharena.ai
Reasoning
68- HLE-DiamondHLE-Diamond: the refined 1,000-question version of Humanity's Last Exam from CAIS and Scale AI, split evenly between reasoning and knowledge questions; accuracy.81 models · in 5 mixes · quality 91 · artificialanalysis.ai
- ARC-AGI-3ARC-AGI-3 semi-private set: interactive game-like environments where the agent must learn rules on the fly; percent of possible score.23 models · in 6 mixes · quality 77 · arcprize.org
- Terminal-Bench-Science-AATerminal-Bench-Science run by AA: 70 expert-curated scientific research workflows across five domains in a terminal; a task passes only if every one of its tests passes. Pass@1 over three repeats.51 models · in 7 mixes · quality 73 · artificialanalysis.ai
- BullshitBench v2Share of 100 nonsense prompts (software, finance, legal, medical, physics) where the model clearly calls out the nonsense instead of confidently answering; 3-judge panel, mean aggregation. Reported per reasoning level.108 models · in 3 mixes · quality 80 · petergpt.github.io
- SimpleBench200+ multiple-choice questions on spatio-temporal reasoning, social intelligence and adversarial trick questions where unspecialized humans outperform frontier models; accuracy averaged over 5 runs.96 models · in 7 mixes · quality 67 · simple-bench.com
- Blueprint-Bench 2Spatial reasoning: reconstruct 2D floor plans from ~20 apartment photos, scored on room connectivity and scale (random baseline normalized to 0).30 models · in 1 mix · quality 72 · andonlabs.com
- Furniture AssemblyEpoch AI's spatial-reasoning benchmark: from photos of IKEA builds (Ställ shoe cabinet, Tonstad bed, Gullaberg dresser) and the official manual, the model decides whether the build is correct and, if not, identifies and describes the mistaken step. Mean score. Data: Epoch AI, CC BY.34 models · in 2 mixes · quality 60 · epoch.ai
- Chess Puzzles (Epoch AI)Epoch AI's chess puzzles: the model finds the best move in positions with a single correct line. Score is the share of puzzles solved. Data: Epoch AI, CC BY.84 models · in 1 mix · quality 65 · epoch.ai
Economics
8- GDP.pdfGDP.pdf: document-heavy professional tasks over PDFs graded against rubric criteria; share of tasks passing all criteria.151 models · in 7 mixes · quality 76 · artificialanalysis.ai
- GDPval-AA v2.1GDPval-AA: economically valuable knowledge-work tasks across occupations, graded pairwise by AA; reported as Elo.209 models · in 5 mixes · quality 64 · artificialanalysis.ai
- Excel Modeling BenchmarkEvaluating agents on Excel-based financial modeling tasks75 models · in 1 mix · quality 61 · vals.ai
- Finance Agent v2Evaluating agents on core financial analyst tasks using the FAB v2 harness79 models · quality 64 · vals.ai
- MortgageTax (Vals)Evaluating reading and understanding tax certificates as images Scores are the values published by VALS AI. Source benchmark version 1.84 models · quality 35 · vals.ai
- TaxEval (Vals)A Vals-created set of questions and responses to tax questions Scores are the values published by VALS AI. Source benchmark version 2.124 models · quality 34 · vals.ai
- CorpFin (Vals)A private benchmark evaluating understanding of long-context credit agreements Scores are the values published by VALS AI. Source benchmark version 2.115 models · quality 31 · vals.ai
- Public Benefits Bench v1 (Vals)Can AI help people navigate SNAP benefits? Scores are the values published by VALS AI. Source benchmark version 1.13 models · quality 21 · vals.ai
Long context
6- MLCR-AAMedical Long Context Reasoning (MLCR-AA): reasoning over long medical records; overall score.102 models · in 6 mixes · quality 74 · artificialanalysis.ai
- AA-LCR v1.1AA-LCR: long-context reasoning over ~100k-token document sets; pass rate.368 models · quality 67 · artificialanalysis.ai
- Arena DocumentHuman preference on document analysis and long-content reasoning; Bradley-Terry rating.37 models · quality 50 · arena.ai
- LongBench v2503 challenging multiple-choice questions with contexts from 8k to 2M words assessing deep understanding and reasoning on realistic long-context multitasks. Maintained by THUDM. Score: Accuracy on 503 multiple-choice questions across six task categories.28 models · quality 48 · longbench2.github.io
- Context Arena — MRCR v2 (8-needle)The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens. Maintained by Context Arena. Score: Cumulative average score up to 128k tokens (unweighted mean of the 8k–128k bins), percent.64 models · quality 41 · contextarena.ai
- RULERNVIDIA synthetic benchmark that evaluates long-context LLMs with configurable sequence length and task complexity across 13 tasks in 4 categories. Maintained by NVIDIA. Score: Per-length scores at 4K-128K plus Avg. across the 13 RULER tasks (wAvg. inc/dec also reported); performance above the 4K Llama-2-7b threshold (85.6%) is underlined.21 models · quality 38 · raw.githubusercontent.com
Agentic
66- AutomationBench-AAAutomationBench-AA: building workflow automations; partial-credit score.152 models · in 5 mixes · quality 76 · artificialanalysis.ai
- Terminal-Bench 4.0Resolution rate of agent+model systems on 66 professional computer-work tasks in a terminal (5 trials per task, 8-hour timeout). Rows are harness+model pairs.23 models · in 8 mixes · quality 68 · tbench.ai
- AA-Briefcase v1.1AA-Briefcase: agentic office-work tasks over real files (xlsx/pptx/docx/pdf) judged pairwise; reported as Elo.156 models · in 9 mixes · quality 64 · artificialanalysis.ai
- Terminal-Bench Science (Vals)Research workflow tasks contributed by practicing scientists across five domains42 models · in 5 mixes · quality 70 · vals.ai
- Agent Arena · 6 categoriesArena.ai Agent Arena: causal 'net improvement' a model contributes to real-world agentic sessions (task success, praise vs complaint, steerability, bash recovery, tool hallucination), estimated from millions of live Agent Mode sessions. Percent with 95% CI; not an Elo.45 models · in 3 mixes · quality 65 · arena.ai
- Terminal-Bench 4.0 (Vals)Vals AI's Terminal-Bench 4.0 aggregate scores from its external-runner leaderboard. Vals publishes a separate system set and score series from the tbench.ai harness-plus-model leaderboard, so the two sources remain distinct.55 models · in 2 mixes · quality 69 · vals.ai
- AutomationBench (Zapier)Zapier's agent benchmark (v1.0.6): end-to-end business workflows across 47 real tools in six functions, graded strictly on the final environment state (every assertion must pass). Private held-out set.40 models · in 2 mixes · quality 68 · zapier.com
- WeirdML v3Fully agentic: 11 hand-made ML and data-analysis tasks run in coding-agent harnesses, where models must explore unfamiliar data and build pipelines with limited data, unspecified goals or little feedback. Overall effective score (95% interval).25 models · in 2 mixes · quality 65 · htihle.github.io
Composite
12- AA Intelligence Index v4.3.2Artificial Analysis' composite intelligence score (v4.3.2), combining 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, and AA-LCR v1.1.445 models · in 2 mixes · quality 70 · artificialanalysis.ai
- Arena Text · 14 categoriesCrowdsourced pairwise human preference on open-ended text prompts (style control on); Bradley-Terry rating.205 models · in 1 mix · quality 59 · arena.ai
- Vals IndexGDP-weighted measure of agentic model performance across finance, coding, and legal tasks.29 models · in 1 mix · quality 59 · vals.ai
- AA Intelligence Index v5.0Artificial Analysis' composite intelligence score (v5.0), combining 9 evaluations: AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 (Hard Subset), Terminal-Bench-Science 0.1.39 models · quality 68 · artificialanalysis.ai
- AA Progress Index v1Artificial Analysis Progress Index: general capability on one scale that holds across years, fitted with item response theory over every evaluation AA has run since late 2022, so models tested on different benchmarks are still comparable. Built for trends over time; the Intelligence Index measures today's frontier.430 models · quality 68 · artificialanalysis.ai
- Vals RSI Index v1.1: Post-training: Finance AgentCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Post-training: Finance Agent score in percentage points.25 models · quality 64 · vals.ai
- Epoch Capabilities IndexEpoch AI's Capabilities Index: a single score per model fitted across many benchmarks (item-response model), so scores are comparable across models tested on different sets. About 100 is GPT-4.1 and about 150 is a current frontier model. Data: Epoch AI, CC BY.180 models · quality 62 · epoch.ai
- Vals RSI Index v1.1: Harness Engineering: JudgeCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Harness Engineering: Judge score in percentage points.25 models · quality 62 · vals.ai
Knowledge
18- Harvey LAB-AA v1.1Harvey LAB-AA v1.1: 120 private agentic legal tasks built by Harvey. Hallucination-Gated All-Pass Rate: a task counts only when a three-judge panel passes every rubric criterion and the deliverables contain no material hallucination. Not comparable with v1.0 scores.24 models · in 1 mix · quality 77 · artificialanalysis.ai
- AA-Omniscience v1.1 · 9 categoriesAA-Omniscience v1.1 Index (-100..100): knowledge accuracy minus a hallucination penalty. v1.1 drops saturated and public questions, adds new ones and revises ambiguous ones; scores run about 2 points below v1.50 models · in 1 mix · quality 65 · artificialanalysis.ai
- MedScribeCan models support doctors with their administrative work?97 models · in 1 mix · quality 59 · vals.ai
- Public Benefits Bench v1.1Accuracy answering questions about SNAP benefits and eligibility.50 models · in 1 mix · quality 58 · vals.ai
- SimpleQA Verified (Epoch AI run)Google DeepMind's cleaned-up SimpleQA short-answer factuality set, run by Epoch AI. Share answered correctly. Data: Epoch AI, CC BY.82 models · quality 64 · epoch.ai
- MedCodeCan models support the medical billing process?95 models · quality 64 · vals.ai
- UGI NatIntA private-question evaluation of general knowledge and reasoning across textbook, popular-culture and world-model topics. Maintained by DontPlanToEnd. Score: NatInt 💡.167 models · quality 61 · huggingface.co
- MMMU-ProMMMU-Pro: multimodal college-level understanding; accuracy.164 models · in 1 mix · quality 48 · artificialanalysis.ai
Creative
22- AA-3DAA-3D: agents in an offline sandbox turn a written brief into an interactive three.js scene; Elo from blind pairwise votes by AA's recruited panel.23 models · in 3 mixes · quality 62 · artificialanalysis.ai
- VoxelBench (text)Models build 3D voxel scenes from text prompts; humans vote head-to-head on the renders. Glicko-2 rating.54 models · in 4 mixes · quality 58 · voxelbench.ai
- MineBenchModels turn a text prompt into raw Minecraft-style block coordinates; humans vote pairwise on the rendered builds. Value is the conservative rankScore (rating − 2·RD), not raw Elo.77 models · in 1 mix · quality 56 · minebench.ai
- threejsevalCoding models build a Three.js scene from the same prompt; humans vote blind on pairs of finished scenes. Value is Bradley-Terry Elo (base 1500), the mean across prompts.30 models · in 1 mix · quality 56 · threejseval.com
- Code2Video-AACode2Video-AA: AA's run of HeyGen's Code2Video Bench. Models turn 114 design briefs into short product and brand videos in HyperFrames HTML; Elo from blind pairwise votes.27 models · in 1 mix · quality 54 · artificialanalysis.ai
- UGI WritingA maintainer-owned writing evaluation considering intelligence, style, repetition and length adherence, informed by human preferences. Maintained by DontPlanToEnd. Score: Writing ✍️.166 models · quality 61 · huggingface.co
- ClutterBench · 1 categoriesClutter as a share of drawn area in generated graphics, using the raw prompt. Red area plus half the orange area, averaged over 6 scored posts. Lower is better.11 models · in 1 mix · quality 48 · clutterbench.adamholter.chatgpt.site
- EQ-Bench Longform Creative WritingLLM-judged benchmark in which models plan and write a short story/novella over 8x 1000-word turns, scored 0-100 across 14 rubric dimensions. Maintained by EQ-Bench. Score: Score (0-100): average of all chapter scores plus the final scored piece across 14 rubric dimensions, with the forced poetry/metaphor criterion weighted 5x and a long-context degradation penalty applied.105 models · quality 57 · eqbench.com
Other
15- T3 Code usage shareShare of all turns sent in T3 Code (2026-09-29 to 2026-10-28), from T3's own telemetry via Slopalytics. Revealed preference among T3 Code users, not a quality measure: price, plans and harness defaults drive it too. Model-level, no reasoning effort.89 models · quality 76 · slopalytics.com
- Arena Vision · 8 categoriesHuman preference on prompts with image inputs (multimodal understanding); Bradley-Terry rating.104 models · in 1 mix · quality 56 · arena.ai
- Hearing BenchModels see only a spectrogram image of a sound clip, never the audio, and must name the sound: everyday sounds, instrument notes, vocal sounds, spoken words and sentences. Strict grading; value is % of 100 clips correct.10 models · quality 55 · hearing-bench.adamholter.chatgpt.site
- AA Openness IndexArtificial Analysis Openness Index (0 to 100): how open a model is, from weight availability and license to transparency about training data and methodology. Not a capability measure.245 models · quality 51 · artificialanalysis.ai
- AGI-Pilled BenchMedian forecast in years after 2026 for 8 AI milestones. Lower means more AGI-pilled. 5 runs per milestone; the IMO control is excluded.14 models · quality 51 · agi-pilled-bench.adamholter.chatgpt.site
- LLM Divergent Thinking Creativity BenchmarkModels generate 25 words maximally distinct from each other and from 50 seed words under starting-letter constraints, scored by LLM-judged word divergence. Maintained by Lech Mazur. Score: Average LLM-judged score of minimum divergences between each generated word and other words, on a 0 to 10 scale (percentage of repeated words also reported).18 models · quality 43 · raw.githubusercontent.com
- AA-PersonalityBench ConvictionAA-PersonalityBench Conviction index: direct, confident, opinionated, and willing to push back. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.10 models · quality 37 · artificialanalysis.ai
- AA-PersonalityBench OrganizationAA-PersonalityBench Organization index: structured, scannable answers that lead with the bottom line. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.10 models · quality 37 · artificialanalysis.ai
Speed
4- Output Speed (tokens/s)Median output tokens per second measured by AA (median across providers, or first-party API).212 models · quality 67 · artificialanalysis.ai
- Latency (Time to First Token)Median seconds from request to first streamed chunk as measured by AA; may be a reasoning token. This is not time to first answer.212 models · quality 67 · artificialanalysis.ai
- Provider latencyMedian provider latency across OpenRouter endpoints, in seconds (3d).180 models · quality 65 · openrouter.ai
- Provider throughputMedian provider output throughput in tokens per second across OpenRouter endpoints (3d).180 models · quality 65 · openrouter.ai
Safety
13- SRE BenchAgent reverse engineering of real-world binary programs without access to source code.34 models · quality 66 · vals.ai
- CWE-Bench-AA Refusal RateShare of CWE-Bench-AA tasks the model refused on safety grounds. Lower means fewer refusals on defensive security work.22 models · in 2 mixes · quality 45 · artificialanalysis.ai
- UGIA private-question suite combining sensitive-topic knowledge with willingness to follow controversial instructions. Maintained by DontPlanToEnd. Score: UGI 🏆.167 models · quality 61 · huggingface.co
- UGI WillingnessA private-question evaluation of how far a model follows challenging instructions before refusing or deviating. Maintained by DontPlanToEnd. Score: W/10 👍.167 models · quality 61 · huggingface.co
- DeepsecBench-AADeepsecBench-AA (based on Vercel's DeepsecBench), part of the AA Cyber Index: finding and fixing vulnerabilities in real codebases; score.24 models · quality 59 · artificialanalysis.ai
- CyberBench v1.1Agent performance reproducing OSS-Fuzz vulnerabilities with proof-of-concept inputs and patching the underlying bugs.44 models · quality 59 · vals.ai
- HalluHardExcludes web-search variants. Score: turn-1 hallucination rate without web search. Collected by Jake Boggs' benchmark dashboard.29 models · quality 53 · halluhard.com
- CWE-BenchCWE-Bench: security-weakness detection; score.22 models · quality 51 · artificialanalysis.ai
Cost
2- AA Intelligence Index Cost per TaskMeasured USD per task, weighted across the Intelligence Index evaluations.35 models · quality 65 · artificialanalysis.ai
- Cost to Run AA Intelligence IndexTotal USD AA spent running the full Intelligence Index evaluation suite for the model (measured input+output token cost, incl. reasoning), not list token price.35 models · quality 65 · artificialanalysis.ai