Run #2705

success · fetched 2026-10-03 03:00:42 · 1.30 MB raw HTML · 171 models

Open this run in comparison view · JSON results

Top quality: Claude Opus 5.5 (Max, Default Fallback) (57.6 pts)

Cost/task vs quality?

171 models · 16 on the Pareto frontier · intelligence quality vs Cost per Task

#?	Pareto?	Model?	Released?	Cost/task?	Time/task?	$/Q?	Qual?	ΔTop?	Intel?	Agent?	Pen?	Score?
1	✓	Claude Opus 5.5 (Max, Default Fallback) Anthropic	2026-09-22	$5.98	13.2m	0.1038	57.6	0.0	57.6	-	27.8	57.6
2		Claude Sonnet 5.5 (Max, Default Fallback) Anthropic	2026-09-28	$7.67	15.0m	0.1369	56.0	-1.6	56.0	-	28.8	56.0
3	✓	Claude Opus 5.5 (Xhigh, Default Fallback) Anthropic	2026-09-22	$3.46	8.35m	0.0618	56.0	-1.6	56.0	-	25.4	56.0
4	✓	Claude Opus 5.5 (High, Default Fallback) Anthropic	2026-09-22	$1.82	4.79m	0.0340	53.6	-4.0	53.6	-	22.6	53.6
5		Claude Fable 5.1 (Max, Default Fallback) Anthropic	2026-09-01	$7.63	12.3m	0.1430	53.4	-4.3	53.4	-	28.8	53.4
6		Claude Fable 5.1 (Xhigh, Default Fallback) Anthropic	2026-09-01	$5.98	10.0m	0.1124	53.2	-4.4	53.2	-	27.8	53.2
7		GPT-6 Astra (Max) OpenAI	2026-09-03	$3.26	8.38m	0.0618	52.7	-4.9	52.7	-	25.1	52.7
8		Gemini 4 Argon (High) Google	2026-09-30	$1.99	-	0.0379	52.6	-5.1	52.6	-	23.0	52.6
9		GPT-6 Astra (Xhigh) OpenAI	2026-09-03	$2.31	5.68m	0.0441	52.4	-5.2	52.4	-	23.6	52.4
10		Claude Sonnet 5.5 (Xhigh, Default Fallback) Anthropic	2026-09-28	$2.75	7.41m	0.0529	51.9	-5.7	51.9	-	24.4	51.9
11	✓	GPT-6.1 Sol (Max) OpenAI	2026-09-29	$0.724	11.6m	0.0140	51.8	-5.8	51.8	-	18.6	51.8
12		Claude Opus 5.5 (Medium, Default Fallback) Anthropic	2026-09-22	$1.34	3.64m	0.0261	51.2	-6.4	51.2	-	21.3	51.2
13		Claude Fable 5.1 (High, Default Fallback) Anthropic	2026-09-01	$3.91	7.21m	0.0765	51.2	-6.5	51.2	-	25.9	51.2
14	✓	GPT-6.1 Sol (Xhigh) OpenAI	2026-09-29	$0.393	5.84m	0.0077	51.0	-6.6	51.0	-	15.9	51.0
15		GPT-6 Astra (High) OpenAI	2026-09-03	$1.73	4.16m	0.0339	50.9	-6.7	50.9	-	22.4	50.9
16		Claude Opus 5 (Max) Anthropic	2026-07-24	$5.86	12.9m	0.1154	50.8	-6.8	50.8	-	27.7	50.8
17	✓	GPT-6.1 Sol (High) OpenAI	2026-09-29	$0.319	4.34m	0.0064	50.2	-7.4	50.2	-	15.0	50.2
18		Claude Opus 5 (Xhigh) Anthropic	2026-07-24	$4.88	11.2m	0.0982	49.7	-7.9	49.7	-	26.9	49.7
19		Claude Fable 5 (Max, Opus 4.8 Fallback) Anthropic	2026-06-09	$8.75	10.4m	0.1762	49.6	-8.0	49.6	-	29.4	49.6
20		GPT-6 Astra (Medium) OpenAI	2026-09-03	$1.54	3.52m	0.0311	49.6	-8.1	49.6	-	21.9	49.6
21		Claude Fable 5.1 (Medium, Default Fallback) Anthropic	2026-09-01	$2.98	5.20m	0.0610	48.9	-8.7	48.9	-	24.7	48.9
22		Claude Opus 5 (High) Anthropic	2026-07-24	$3.61	8.67m	0.0751	48.1	-9.5	48.1	-	25.6	48.1
23		Muse Spark 1.3 (Max) Meta	2026-09-02	$1.60	7.64m	0.0334	48.1	-9.5	48.1	-	22.1	48.1
24	✓	GPT-6.1 Sol (Medium) OpenAI	2026-09-29	$0.214	2.95m	0.0045	47.8	-9.8	47.8	-	13.3	47.8
25		GPT-6 Sol (Max) OpenAI	2026-09-22	$1.04	5.38m	0.0219	47.6	-10.0	47.6	-	20.2	47.6
26		GPT-5.6 Sol (Max) OpenAI	2026-07-09	$1.99	6.22m	0.0423	47.0	-10.6	47.0	-	23.0	47.0
27		Claude Fable 5.1 (Low, Default Fallback) Anthropic	2026-09-01	$2.37	4.22m	0.0506	46.8	-10.8	46.8	-	23.7	46.8
28		Claude Sonnet 5.5 (High, Default Fallback) Anthropic	2026-09-28	$1.12	3.84m	0.0240	46.8	-10.9	46.8	-	20.5	46.8
29		Grok 4.7 (Xhigh) SpaceXAI	2026-09-21	$3.74	18.2m	0.0805	46.4	-11.2	46.4	-	25.7	46.4
30		Grok 4.7 (High) SpaceXAI	2026-09-21	$2.73	14.3m	0.0588	46.3	-11.3	46.3	-	24.4	46.3
31	✓	MiMo-V2.6-Pro Xiaomi	2026-09-21	$0.133	23.3m	0.0029	46.3	-11.3	46.3	-	11.2	46.3
32		GPT-6 Astra (Low) OpenAI	2026-09-03	$0.818	1.56m	0.0179	45.8	-11.8	45.8	-	19.1	45.8
33		Qwen3.8 Max (0902) Alibaba	2026-09-02	$5.41	36.7m	0.1191	45.4	-12.2	45.4	-	27.3	45.4
34		Muse Spark 1.3 (Xhigh) Meta	2026-09-02	$1.37	6.24m	0.0303	45.1	-12.5	45.1	-	21.4	45.1
35		Claude Opus 5 (Medium) Anthropic	2026-07-24	$2.19	5.30m	0.0488	44.8	-12.8	44.8	-	23.4	44.8
36		GLM-5.3 (Max) Z AI	2026-08-18	$2.01	12.8m	0.0448	44.8	-12.8	44.8	-	23.0	44.8
37		Grok 4.6 (High) SpaceXAI	2026-08-12	$1.86	8.11m	0.0420	44.3	-13.3	44.3	-	22.7	44.3
38		GPT-6 Sol (Xhigh) OpenAI	2026-09-22	$0.521	2.86m	0.0118	44.2	-13.4	44.2	-	17.2	44.2
39		Grok 4.6 (Xhigh) SpaceXAI	2026-08-12	$2.32	9.32m	0.0526	44.2	-13.4	44.2	-	23.7	44.2
40		GPT-5.6 Sol (Xhigh) OpenAI	2026-07-09	$1.18	4.31m	0.0269	44.0	-13.6	44.0	-	20.7	44.0
41		Step 5 Preview StepFun	2026-09-18	$0.717	11.5m	0.0164	43.7	-13.9	43.7	-	18.6	43.7
42		Kimi K3 (Max) Kimi	2026-07-16	$2.00	19.7m	0.0459	43.6	-14.0	43.6	-	23.0	43.6
43		Grok 4.6 (Medium) SpaceXAI	2026-08-12	$1.50	7.30m	0.0349	42.8	-14.8	42.8	-	21.8	42.8
44		GPT-6 Sol (High) OpenAI	2026-09-22	$0.373	1.90m	0.0088	42.4	-15.2	42.4	-	15.7	42.4
45		GPT-5.6 Sol (High) OpenAI	2026-07-09	$0.808	2.99m	0.0191	42.3	-15.3	42.3	-	19.1	42.3
46		Claude Opus 5.5 (Low, Default Fallback) Anthropic	2026-09-22	$0.551	1.39m	0.0130	42.3	-15.3	42.3	-	17.4	42.3
47		Grok 4.7 (Low) SpaceXAI	2026-09-21	$1.25	6.38m	0.0296	42.2	-15.4	42.2	-	21.0	42.2
48	✓	GPT-6.1 Sol (Low) OpenAI	2026-09-29	$0.131	1.39m	0.0031	42.1	-15.5	42.1	-	11.2	42.1
49		GPT-5.6 Terra (Max) OpenAI	2026-07-09	$1.40	6.25m	0.0332	42.1	-15.5	42.1	-	21.5	42.1
50		GLM 5.3 Flash Z AI	2026-08-26	$0.253	16.5m	0.0061	41.8	-15.8	41.8	-	14.0	41.8
51		Claude Opus 4.8 (Max) Anthropic	2026-05-28	$4.08	11.6m	0.0977	41.8	-15.8	41.8	-	26.1	41.8
52		Gemini 3.8 Flash (High) Google	2026-09-02	$1.24	4.83m	0.0304	40.9	-16.7	40.9	-	20.9	40.9
53		Claude Sonnet 5.5 (Medium, Default Fallback) Anthropic	2026-09-28	$0.589	2.26m	0.0144	40.8	-16.8	40.8	-	17.7	40.8
54		Qwen3.8 Max Alibaba	2026-08-03	$2.67	21.4m	0.0665	40.2	-17.5	40.2	-	24.3	40.2
55		Qwen3.8 2.4T A95B Alibaba	2026-08-12	$2.16	23.5m	0.0541	39.9	-17.7	39.9	-	23.3	39.9
56		Qwen3.8-Flash-Next Alibaba	2026-08-26	$0.372	24.5m	0.0093	39.8	-17.8	39.8	-	15.7	39.8
57		GPT-6 Sol (Medium) OpenAI	2026-09-22	$0.247	-	0.0062	39.8	-17.8	39.8	-	13.9	39.8
58		Gemini 3.8 Flash (Medium) Google	2026-09-02	$0.931	-	0.0234	39.8	-17.8	39.8	-	19.7	39.8
59		Muse Spark 1.2 (Xhigh) Meta	2026-08-05	$0.975	3.24m	0.0246	39.6	-18.0	39.6	-	19.9	39.6
60		DeepSeek V4.1 Flash (Max) DeepSeek	2026-09-10	$0.265	5.08m	0.0067	39.5	-18.2	39.5	-	14.2	39.5
61		Claude Opus 5 (Low) Anthropic	2026-07-24	$1.10	2.81m	0.0279	39.4	-18.3	39.4	-	20.4	39.4
62		GPT-5.6 Sol (Medium) OpenAI	2026-07-09	$0.505	1.82m	0.0129	39.2	-18.4	39.2	-	17.0	39.2
63		Gemini 3.7 Flash (High) Google	2026-08-13	$0.925	3.29m	0.0237	39.1	-18.6	39.1	-	19.7	39.1
64		Grok 4.5 (High) SpaceXAI	2026-07-08	$1.04	8.42m	0.0267	38.8	-18.8	38.8	-	20.2	38.8
65		GPT-5.5 (Xhigh) OpenAI	2026-04-23	$2.63	4.71m	0.0687	38.4	-19.3	38.4	-	24.2	38.4
66		Claude Sonnet 5 (Max) Anthropic	2026-06-30	$5.09	13.9m	0.1334	38.2	-19.5	38.2	-	27.1	38.2
67	✓	GPT-6 Luna (Max) OpenAI	2026-09-22	$0.0678	6.34m	0.0018	38.1	-19.5	38.1	-	8.3	38.1
68		GPT-5.6 Terra (Xhigh) OpenAI	2026-07-09	$0.632	3.75m	0.0166	38.0	-19.7	38.0	-	18.0	38.0
69	✓	MiMo-V2.6-Flash Xiaomi	2026-09-21	$0.0623	26.2m	0.0016	37.9	-19.7	37.9	-	7.9	37.9
70		GPT-5.6 Luna (Max) OpenAI	2026-07-09	$0.178	5.38m	0.0048	37.3	-20.3	37.3	-	12.5	37.3
71		GPT-5.5 (High) OpenAI	2026-04-23	$1.54	3.03m	0.0417	37.0	-20.6	37.0	-	21.9	37.0
72		DeepSeek V4 Pro 0813 (Max) DeepSeek	2026-08-13	$0.674	7.31m	0.0187	36.0	-21.6	36.0	-	18.3	36.0
73		Claude Sonnet 5.5 (Low, Default Fallback) Anthropic	2026-09-28	$0.417	1.57m	0.0116	35.9	-21.8	35.9	-	16.2	35.9
74		Grok 4.6 (Low) SpaceXAI	2026-08-12	$0.475	2.70m	0.0135	35.1	-22.5	35.1	-	16.8	35.1
75		DeepSeek V4 Flash Vision (Max) DeepSeek	2026-08-21	$0.314	3.96m	0.0090	34.8	-22.8	34.8	-	15.0	34.8
76	✓	GPT-6 Luna (Xhigh) OpenAI	2026-09-22	$0.0422	3.30m	0.0012	34.6	-23.1	34.6	-	6.3	34.6
77		GPT-5.6 Luna (Xhigh) OpenAI	2026-07-09	$0.0853	3.37m	0.0025	34.6	-23.1	34.6	-	9.3	34.6
78		Claude Sonnet 5 (Xhigh) Anthropic	2026-06-30	$2.87	8.54m	0.0835	34.4	-23.2	34.4	-	24.6	34.4
79		DeepSeek V4 Flash 0731 (Max) DeepSeek	2026-07-31	$0.220	3.75m	0.0064	34.3	-23.3	34.3	-	13.4	34.3
80		GLM-5.3 (Low) Z AI	2026-08-18	$0.852	7.02m	0.0248	34.3	-23.3	34.3	-	19.3	34.3
81		GPT-5.6 Terra (High) OpenAI	2026-07-09	$0.338	2.19m	0.0099	34.2	-23.4	34.2	-	15.3	34.2
82		GPT-6 Sol (Low) OpenAI	2026-09-22	$0.133	42s	0.0039	34.2	-23.5	34.2	-	11.2	34.2
83		Gemini 3.6 Flash (High) Google	2026-07-21	$0.929	3.30m	0.0273	34.0	-23.6	34.0	-	19.7	34.0
84		GPT-5.5 (Medium) OpenAI	2026-04-23	$0.902	1.74m	0.0267	33.8	-23.8	33.8	-	19.6	33.8
85		Muse Spark 1.1 (Xhigh) Meta	2026-07-09	$1.38	7.82m	0.0409	33.7	-23.9	33.7	-	21.4	33.7
86		GLM-5.2 (Max) Z AI	2026-06-16	$1.47	10.2m	0.0437	33.7	-23.9	33.7	-	21.7	33.7
87		Qwen3.8 27B (Xhigh) Alibaba	2026-08-14	$1.01	19.0m	0.0299	33.7	-23.9	33.7	-	20.0	33.7
88		GPT-5.6 Sol (Low) OpenAI	2026-07-09	$0.261	57s	0.0078	33.5	-24.1	33.5	-	14.2	33.5
89	✓	GPT-6 Luna (High) OpenAI	2026-09-22	$0.0290	2.41m	0.0009	32.9	-24.7	32.9	-	4.6	32.9
90		Gemini 3.5 Flash (High) Google	2026-05-19	$1.56	4.16m	0.0479	32.6	-25.0	32.6	-	21.9	32.6
91		GPT-5.6 Luna (High) OpenAI	2026-07-09	$0.0440	2.06m	0.0014	32.1	-25.5	32.1	-	6.4	32.1
92		Claude Sonnet 5 (High) Anthropic	2026-06-30	$1.79	6.21m	0.0566	31.7	-26.0	31.7	-	22.5	31.7
93		DeepSeek V4 Pro 0424 (Max) DeepSeek	2026-04-24	$0.122	6.36m	0.0040	30.4	-27.2	30.4	-	10.8	30.4
94		GPT-5.6 Terra (Medium) OpenAI	2026-07-09	$0.183	1.05m	0.0061	30.1	-27.5	30.1	-	12.6	30.1
95		Kimi K3 (Low) Kimi	2026-07-16	$1.15	5.47m	0.0381	30.1	-27.6	30.1	-	20.6	30.1
96		Claude Sonnet 4.6 (Max) Anthropic	2026-02-17	$2.49	20.1m	0.0827	30.1	-27.6	30.1	-	24.0	30.1
97	✓	GPT-6 Luna (Medium) OpenAI	2026-09-22	$0.0175	-	0.0006	29.9	-27.7	29.9	-	2.4	29.9
98		Gemini 3.1 Pro Preview Google	2026-02-19	$0.675	2.62m	0.0227	29.7	-27.9	29.7	-	18.3	29.7
99		Qwen3.7 Max Alibaba	2026-05-19	$1.15	2.18m	0.0389	29.5	-28.2	29.5	-	20.6	29.5
100		MiniMax-M3 MiniMax	2026-06-01	$0.508	10.1m	0.0174	29.2	-28.4	29.2	-	17.1	29.2