SRE Bench

Agent reverse engineering of real-world binary programs without access to source code.

29 models · 29 results · updated 2026-10-14 · vals.ai ↗

More from this benchmark

1 published measures

Time per task35 results · lower is better

19 more in settings

29 rows

#	Model	Effort	SRE Bench	Subscription $ / task	API $ / task	Time
1	GPT-6 Astra	max	56.9%	$0.88	$10	18 min
2	GPT-6.1 Sol	max	50.8%	$0.31	$2.69	33 min
3	Gemini 4 Argon	high	44.3%	$11	$30	1.6 h
4	Opus 5.5	max	33.6%	$0.52	$35	1.6 h
5	GPT-5.6 Sol	max	30.5%	$2.35	$43	1.5 h
6	GPT-6 Sol	max	30.5%	$1.28	$21	1.1 h
7	Sonnet 5.5	max	30.2%	$0.40	$27	1.9 h
8	Fable 5.1	max	22.9%	$5.02	$32	2.0 h
9	Opus 5	max	12.2%	$0.41	$24	1.3 h
10	Gemini 3.8 Flash	high	11.1%	$1.11	$27	2.5 h
11	GPT-5.6 Terra	max	9.5%	$1.70	$31	3.3 h
12	GLM 5.3	max	9.2%	$2.85	$27	2.8 h
13	Haiku 5.5	max	5.7%	$0.067	$3.14	58 min
14	Gemini 3.7 Flash	high	4.6%	$2.77	$21	1.8 h
15	MiMo V2.6 Flash	high	4.2%	free	$0.40	3.4 h
16	GPT-5.5	xhigh	3.8%	$0.98	$18	46 min
17	Grok 4.7	xhigh	3.8%	$6.01	$65	1.7 h
18	Muse Spark 1.3	max	3.8%	$13	$26	1.8 h
19	MiMo V2.6 Pro	high	3.1%	$0.45	$0.53	2.4 h
20	GPT-6 Luna	max	2.7%	$0.23	$1.55	1.4 h
21	DeepSeek V4 Pro 0813	max	1.9%	$5.58	$5.58	2.3 h
22	GPT-5.6 Luna	max	1.5%	$0.32	$3.44	1.7 h
23	DeepSeek V4.1 Flash	max	0.8%	$0.034	$0.55	1.0 h
24	GLM 5.3 Flash	max	0.8%	free	$2.29	3.2 h
25	Grok 4.5	high	0.8%	$2.10	$13	1.1 h
26	Sonnet 5	max	0.4%	$0.32	$18	3.9 h
27	GLM 5.2	max	0.0%	$2.32	$27	2.5 h
28	Inkling	max	0.0%	free	$2.23	29 min
29	Muse Spark 1.2	xhigh	0.0%	$12	$23	1.7 h