Run #3287

success · fetched 2026-10-28 11:00:56 · 1.52 MB raw HTML · 72 models

Open this run in comparison view · JSON results

Top quality: Claude Opus 5.5 (Max, Default Fallback) (58.4 pts)

Cost/task vs quality?

72 models · 16 on the Pareto frontier · intelligence quality vs Cost per Task

318324762$0.01$0.10$1$10 cost per task, log scale intelligence quality GPT-6 Astra (Max) · Cost/task: $6.24 · Time/task: 11.9m · Quality: 56.9Claude Sonnet 5.5 (Max, Default Fallback) · Cost/task: $10.0 · Time/task: 22.1m · Quality: 54.2Claude Opus 5.5 (High, Default Fallback) · Cost/task: $3.46 · Time/task: 7.73m · Quality: 51.7GPT-6 Astra (Medium) · Cost/task: $2.73 · Time/task: 5.06m · Quality: 51.5GPT-6.1 Terra (Max) · Cost/task: $0.961 · Time/task: 10.9m · Quality: 51.0Gemini 4 Argon (High) · Cost/task: $3.84 · Time/task: 22.8m · Quality: 50.4Claude Fable 5.1 (Xhigh, Default Fallback) · Cost/task: $10.9 · Time/task: 15.8m · Quality: 49.7Claude Sonnet 5.5 (Xhigh, Default Fallback) · Cost/task: $4.09 · Time/task: 11.5m · Quality: 49.6Claude Fable 5.1 (Max, Default Fallback) · Cost/task: $13.0 · Time/task: 19.6m · Quality: 49.0Claude Opus 5.5 (Medium, Default Fallback) · Cost/task: $2.45 · Time/task: 5.61m · Quality: 48.1Claude Fable 5.1 (High, Default Fallback) · Cost/task: $7.56 · Time/task: 11.2m · Quality: 47.0Kimi K3.1 Pro · Cost/task: $3.68 · Time/task: 25.1m · Quality: 46.0GPT-6.1 Terra (Xhigh) · Cost/task: $0.401 · Time/task: 4.78m · Quality: 45.0Muse Wonder (Max) · Cost/task: $6.95 · Time/task: 7.80m · Quality: 43.3GPT-6 Astra (Low) · Cost/task: $1.42 · Time/task: 3.15m · Quality: 43.0Claude Fable 5.1 (Medium, Default Fallback) · Cost/task: $5.58 · Time/task: 8.38m · Quality: 43.0Claude Sonnet 5.5 (High, Default Fallback) · Cost/task: $1.64 · Time/task: 5.53m · Quality: 41.1Claude Fable 5.1 (Low, Default Fallback) · Cost/task: $4.49 · Time/task: 6.85m · Quality: 38.5Muse Spark 1.3 (Max) · Cost/task: $2.39 · Time/task: 7.46m · Quality: 36.4Claude Haiku 5.5 (Max) · Cost/task: $0.411 · Time/task: 9.89m · Quality: 36.2Grok 4.7 (High) · Cost/task: $6.18 · Time/task: 15.9m · Quality: 35.0Claude Opus 5.5 (Low, Default Fallback) · Cost/task: $0.976 · Time/task: 2.45m · Quality: 34.5Claude Haiku 5.5 (Xhigh) · Cost/task: $0.248 · Time/task: 7.35m · Quality: 34.3Grok 4.7 (Xhigh) · Cost/task: $7.51 · Time/task: 16.5m · Quality: 34.3Qwen3.8 Max (0902) · Cost/task: $10.0 · Time/task: 52.3m · Quality: 33.6DeepSeek V4.1 Flash 1017 (Max) · Cost/task: $0.602 · Time/task: 8.72m · Quality: 32.5GLM-5.3 (Max) · Cost/task: $5.01 · Time/task: 17.7m · Quality: 31.7Grok 4.7 (Low) · Cost/task: $2.92 · Time/task: 8.97m · Quality: 30.9Gemini 3.8 Flash (High) · Cost/task: $7.77 · Time/task: 12.1m · Quality: 30.5GPT-6.1 Terra (Non-reasoning) · Cost/task: $0.226 · Time/task: 1.73m · Quality: 30.4Claude Sonnet 5.5 (Medium, Default Fallback) · Cost/task: $0.781 · Time/task: 2.96m · Quality: 30.3Gemini 3.8 Flash (Medium) · Cost/task: $6.26 · Time/task: - · Quality: 30.0MiMo-V2.6-Pro · Cost/task: $0.257 · Time/task: 30.5m · Quality: 29.9Ling 3.1 Flash 1019 · Cost/task: $3.19 · Time/task: - · Quality: 29.5MiniMax-M3.1-Flash · Cost/task: $0.491 · Time/task: 16.9m · Quality: 28.9Step 5 Preview · Cost/task: $2.35 · Time/task: 9.46m · Quality: 28.5GPT-6 Luna (Max) · Cost/task: $0.152 · Time/task: 9.96m · Quality: 27.8MiMo-V2.6-Flash · Cost/task: $0.129 · Time/task: 37.0m · Quality: 26.7Claude Sonnet 5.5 (Low, Default Fallback) · Cost/task: $0.511 · Time/task: 2.01m · Quality: 25.3Mistral Large 4 Preview · Cost/task: $5.54 · Time/task: 17.8m · Quality: 25.3Muse Spark Open (Xhigh) · Cost/task: $2.02 · Time/task: 5.79m · Quality: 24.7Claude Haiku 5.5 (Medium) · Cost/task: $0.0918 · Time/task: 3.22m · Quality: 24.6GPT-6 Luna (Xhigh) · Cost/task: $0.104 · Time/task: 5.95m · Quality: 23.3DeepSeek V4 Pro 0813 (Max) · Cost/task: $1.77 · Time/task: 12.8m · Quality: 23.1GLM-5.3 (Low) · Cost/task: $1.57 · Time/task: 9.60m · Quality: 23.1GPT-6 Luna (High) · Cost/task: $0.0729 · Time/task: 4.49m · Quality: 21.4Qwen3.8 27B (Xhigh) · Cost/task: $3.53 · Time/task: 34.1m · Quality: 20.1MiniMax-M3 · Cost/task: $1.29 · Time/task: 11.1m · Quality: 18.7Beam 1.0 · Cost/task: $0.284 · Time/task: 1.01m · Quality: 17.7Nemotron 3.5 Super VL · Cost/task: $0.557 · Time/task: 2.64m · Quality: 13.6Inkling Small · Cost/task: $0.203 · Time/task: 2.46m · Quality: 12.9Inkling (Xhigh) · Cost/task: $1.35 · Time/task: 3.45m · Quality: 12.5Nemotron 3 Ultra 550B A55B (Reasoning) · Cost/task: $2.23 · Time/task: 4.23m · Quality: 11.8Gemini 3.5 Flash-Lite · Cost/task: $0.693 · Time/task: 1.08m · Quality: 11.7GPT-6 Luna (Non-reasoning) · Cost/task: $0.0714 · Time/task: 1.44m · Quality: 9.4Muse Glimmer (High) · Cost/task: $0.161 · Time/task: 3.50m · Quality: 6.3Claude Opus 5.5 (Max, Default Fallback) · Cost/task: $10.0 · Time/task: 18.9m · Quality: 58.4 · Pareto frontierClaude Opus 5.5 (Xhigh, Default Fallback) · Cost/task: $6.05 · Time/task: 13.0m · Quality: 57.0 · Pareto frontierGPT-6 Astra (Xhigh) · Cost/task: $3.66 · Time/task: 6.52m · Quality: 55.3 · Pareto frontierGPT-6 Astra (High) · Cost/task: $3.24 · Time/task: 6.21m · Quality: 54.8 · Pareto frontierGPT-6.1 Sol (Max) · Cost/task: $1.27 · Time/task: 14.3m · Quality: 54.5 · Pareto frontierGPT-6.1 Sol (Xhigh) · Cost/task: $0.764 · Time/task: 7.27m · Quality: 52.3 · Pareto frontierGPT-6.1 Sol (High) · Cost/task: $0.591 · Time/task: 5.37m · Quality: 50.2 · Pareto frontierGPT-6.1 Sol (Medium) · Cost/task: $0.359 · Time/task: 3.85m · Quality: 46.1 · Pareto frontierGPT-6.1 Terra (High) · Cost/task: $0.259 · Time/task: 3.04m · Quality: 42.3 · Pareto frontierGPT-6.1 Terra (Medium) · Cost/task: $0.169 · Time/task: 1.97m · Quality: 38.1 · Pareto frontierGPT-6.1 Sol (Low) · Cost/task: $0.161 · Time/task: 1.73m · Quality: 34.5 · Pareto frontierClaude Haiku 5.5 (High) · Cost/task: $0.159 · Time/task: 4.97m · Quality: 29.6 · Pareto frontierGPT-6.1 Terra (Low) · Cost/task: $0.0713 · Time/task: 48s · Quality: 29.2 · Pareto frontierClaude Haiku 5.5 (Low) · Cost/task: $0.0438 · Time/task: 1.63m · Quality: 18.6 · Pareto frontierGPT-6 Luna (Medium) · Cost/task: $0.0422 · Time/task: - · Quality: 18.3 · Pareto frontierGPT-6 Luna (Low) · Cost/task: $0.00512 · Time/task: 28s · Quality: 9.7 · Pareto frontier GPT-6 Luna (low)GPT-6 Luna (mediu…Claude Haiku 5.5 …GPT-6.1 Terra (lo…Claude Haiku 5.5 …GPT-6.1 Sol (low)GPT-6.1 Terra (me…GPT-6.1 Terra (hi…GPT-6.1 Sol (medi…GPT-6.1 Sol (high)GPT-6.1 Sol (xhig…GPT-6.1 Sol (max)GPT-6 Astra (high)GPT-6 Astra (xhig…Claude Opus 5.5 (…Claude Opus 5.5 (…
#?Pareto?Model?Released?Cost/task?Time/task?$/Q?Qual?ΔTop?Intel?Agent?Pen?Score?
1 ✓ Claude Opus 5.5 (Max, Default Fallback)
Anthropic		 2026-09-22 $10.0 18.9m 0.1718 58.4 0.0 58.4 - 30.0 58.4
2 ✓ Claude Opus 5.5 (Xhigh, Default Fallback)
Anthropic		 2026-09-22 $6.05 13.0m 0.1061 57.0 -1.4 57.0 - 27.8 57.0
3 GPT-6 Astra (Max)
OpenAI		 2026-09-03 $6.24 11.9m 0.1098 56.9 -1.6 56.9 - 28.0 56.9
4 ✓ GPT-6 Astra (Xhigh)
OpenAI		 2026-09-03 $3.66 6.52m 0.0662 55.3 -3.2 55.3 - 25.6 55.3
5 ✓ GPT-6 Astra (High)
OpenAI		 2026-09-03 $3.24 6.21m 0.0590 54.8 -3.6 54.8 - 25.1 54.8
6 ✓ GPT-6.1 Sol (Max)
OpenAI		 2026-09-29 $1.27 14.3m 0.0233 54.5 -3.9 54.5 - 21.0 54.5
7 Claude Sonnet 5.5 (Max, Default Fallback)
Anthropic		 2026-09-28 $10.0 22.1m 0.1847 54.2 -4.2 54.2 - 30.0 54.2
8 ✓ GPT-6.1 Sol (Xhigh)
OpenAI		 2026-09-29 $0.764 7.27m 0.0146 52.3 -6.1 52.3 - 18.8 52.3
9 Claude Opus 5.5 (High, Default Fallback)
Anthropic		 2026-09-22 $3.46 7.73m 0.0670 51.7 -6.7 51.7 - 25.4 51.7
10 GPT-6 Astra (Medium)
OpenAI		 2026-09-03 $2.73 5.06m 0.0531 51.5 -7.0 51.5 - 24.4 51.5
11 GPT-6.1 Terra (Max)
OpenAI		 2026-10-26 $0.961 10.9m 0.0189 51.0 -7.4 51.0 - 19.8 51.0
12 Gemini 4 Argon (High)
Google		 2026-09-30 $3.84 22.8m 0.0762 50.4 -8.0 50.4 - 25.8 50.4
13 ✓ GPT-6.1 Sol (High)
OpenAI		 2026-09-29 $0.591 5.37m 0.0118 50.2 -8.2 50.2 - 17.7 50.2
14 Claude Fable 5.1 (Xhigh, Default Fallback)
Anthropic		 2026-09-01 $10.9 15.8m 0.2191 49.7 -8.7 49.7 - 30.4 49.7
15 Claude Sonnet 5.5 (Xhigh, Default Fallback)
Anthropic		 2026-09-28 $4.09 11.5m 0.0825 49.6 -8.8 49.6 - 26.1 49.6
16 Claude Fable 5.1 (Max, Default Fallback)
Anthropic		 2026-09-01 $13.0 19.6m 0.2661 49.0 -9.4 49.0 - 31.2 49.0
17 Claude Opus 5.5 (Medium, Default Fallback)
Anthropic		 2026-09-22 $2.45 5.61m 0.0510 48.1 -10.3 48.1 - 23.9 48.1
18 Claude Fable 5.1 (High, Default Fallback)
Anthropic		 2026-09-01 $7.56 11.2m 0.1608 47.0 -11.4 47.0 - 28.8 47.0
19 ✓ GPT-6.1 Sol (Medium)
OpenAI		 2026-09-29 $0.359 3.85m 0.0078 46.1 -12.3 46.1 - 15.6 46.1
20 Kimi K3.1 Pro
Kimi		 2026-10-23 $3.68 25.1m 0.0800 46.0 -12.4 46.0 - 25.7 46.0
21 GPT-6.1 Terra (Xhigh)
OpenAI		 2026-10-26 $0.401 4.78m 0.0089 45.0 -13.5 45.0 - 16.0 45.0
22 Muse Wonder (Max)
Meta		 2026-10-12 $6.95 7.80m 0.1605 43.3 -15.1 43.3 - 28.4 43.3
23 GPT-6 Astra (Low)
OpenAI		 2026-09-03 $1.42 3.15m 0.0331 43.0 -15.4 43.0 - 21.5 43.0
24 Claude Fable 5.1 (Medium, Default Fallback)
Anthropic		 2026-09-01 $5.58 8.38m 0.1299 43.0 -15.4 43.0 - 27.5 43.0
25 ✓ GPT-6.1 Terra (High)
OpenAI		 2026-10-26 $0.259 3.04m 0.0061 42.3 -16.1 42.3 - 14.1 42.3
26 Claude Sonnet 5.5 (High, Default Fallback)
Anthropic		 2026-09-28 $1.64 5.53m 0.0398 41.1 -17.3 41.1 - 22.1 41.1
27 Claude Fable 5.1 (Low, Default Fallback)
Anthropic		 2026-09-01 $4.49 6.85m 0.1166 38.5 -19.9 38.5 - 26.5 38.5
28 ✓ GPT-6.1 Terra (Medium)
OpenAI		 2026-10-26 $0.169 1.97m 0.0044 38.1 -20.3 38.1 - 12.3 38.1
29 Muse Spark 1.3 (Max)
Meta		 2026-09-02 $2.39 7.46m 0.0655 36.4 -22.0 36.4 - 23.8 36.4
30 Claude Haiku 5.5 (Max)
Anthropic		 2026-10-07 $0.411 9.89m 0.0113 36.2 -22.2 36.2 - 16.1 36.2
31 Grok 4.7 (High)
SpaceXAI		 2026-09-21 $6.18 15.9m 0.1766 35.0 -23.4 35.0 - 27.9 35.0
32 ✓ GPT-6.1 Sol (Low)
OpenAI		 2026-09-29 $0.161 1.73m 0.0047 34.5 -23.9 34.5 - 12.1 34.5
33 Claude Opus 5.5 (Low, Default Fallback)
Anthropic		 2026-09-22 $0.976 2.45m 0.0283 34.5 -23.9 34.5 - 19.9 34.5
34 Claude Haiku 5.5 (Xhigh)
Anthropic		 2026-10-07 $0.248 7.35m 0.0072 34.3 -24.1 34.3 - 14.0 34.3
35 Grok 4.7 (Xhigh)
SpaceXAI		 2026-09-21 $7.51 16.5m 0.2188 34.3 -24.1 34.3 - 28.8 34.3
36 Qwen3.8 Max (0902)
Alibaba		 2026-09-02 $10.0 52.3m 0.2987 33.6 -24.9 33.6 - 30.0 33.6
37 DeepSeek V4.1 Flash 1017 (Max)
DeepSeek		 2026-10-17 $0.602 8.72m 0.0185 32.5 -25.9 32.5 - 17.8 32.5
38 GLM-5.3 (Max)
Z AI		 2026-08-18 $5.01 17.7m 0.1583 31.7 -26.8 31.7 - 27.0 31.7
39 Grok 4.7 (Low)
SpaceXAI		 2026-09-21 $2.92 8.97m 0.0946 30.9 -27.5 30.9 - 24.7 30.9
40 Gemini 3.8 Flash (High)
Google		 2026-09-02 $7.77 12.1m 0.2542 30.5 -27.9 30.5 - 28.9 30.5
41 GPT-6.1 Terra (Non-reasoning)
OpenAI		 2026-10-26 $0.226 1.73m 0.0074 30.4 -28.0 30.4 - 13.5 30.4
42 Claude Sonnet 5.5 (Medium, Default Fallback)
Anthropic		 2026-09-28 $0.781 2.96m 0.0257 30.3 -28.1 30.3 - 18.9 30.3
43 Gemini 3.8 Flash (Medium)
Google		 2026-09-02 $6.26 - 0.2088 30.0 -28.4 30.0 - 28.0 30.0
44 MiMo-V2.6-Pro
Xiaomi		 2026-09-21 $0.257 30.5m 0.0086 29.9 -28.5 29.9 - 14.1 29.9
45 ✓ Claude Haiku 5.5 (High)
Anthropic		 2026-10-07 $0.159 4.97m 0.0054 29.6 -28.8 29.6 - 12.0 29.6
46 Ling 3.1 Flash 1019
InclusionAI		 2026-10-22 $3.19 - 0.1081 29.5 -28.9 29.5 - 25.0 29.5
47 ✓ GPT-6.1 Terra (Low)
OpenAI		 2026-10-26 $0.0713 48s 0.0024 29.2 -29.2 29.2 - 8.5 29.2
48 MiniMax-M3.1-Flash
MiniMax		 2026-10-19 $0.491 16.9m 0.0170 28.9 -29.5 28.9 - 16.9 28.9
49 Step 5 Preview
StepFun		 2026-09-18 $2.35 9.46m 0.0822 28.5 -29.9 28.5 - 23.7 28.5
50 GPT-6 Luna (Max)
OpenAI		 2026-09-22 $0.152 9.96m 0.0055 27.8 -30.7 27.8 - 11.8 27.8
51 MiMo-V2.6-Flash
Xiaomi		 2026-09-21 $0.129 37.0m 0.0048 26.7 -31.7 26.7 - 11.1 26.7
52 Claude Sonnet 5.5 (Low, Default Fallback)
Anthropic		 2026-09-28 $0.511 2.01m 0.0202 25.3 -33.1 25.3 - 17.1 25.3
53 Mistral Large 4 Preview
Mistral		 2026-10-06 $5.54 17.8m 0.2190 25.3 -33.1 25.3 - 27.4 25.3
54 Muse Spark Open (Xhigh)
Meta		 2026-10-15 $2.02 5.79m 0.0817 24.7 -33.7 24.7 - 23.0 24.7
55 Claude Haiku 5.5 (Medium)
Anthropic		 2026-10-07 $0.0918 3.22m 0.0037 24.6 -33.9 24.6 - 9.6 24.6
56 GPT-6 Luna (Xhigh)
OpenAI		 2026-09-22 $0.104 5.95m 0.0045 23.3 -35.1 23.3 - 10.2 23.3
57 DeepSeek V4 Pro 0813 (Max)
DeepSeek		 2026-08-13 $1.77 12.8m 0.0767 23.1 -35.3 23.1 - 22.5 23.1
58 GLM-5.3 (Low)
Z AI		 2026-08-18 $1.57 9.60m 0.0683 23.1 -35.3 23.1 - 22.0 23.1
59 GPT-6 Luna (High)
OpenAI		 2026-09-22 $0.0729 4.49m 0.0034 21.4 -37.0 21.4 - 8.6 21.4
60 Qwen3.8 27B (Xhigh)
Alibaba		 2026-08-14 $3.53 34.1m 0.1755 20.1 -38.3 20.1 - 25.5 20.1
61 MiniMax-M3
MiniMax		 2026-06-01 $1.29 11.1m 0.0686 18.7 -39.7 18.7 - 21.1 18.7
62 ✓ Claude Haiku 5.5 (Low)
Anthropic		 2026-10-07 $0.0438 1.63m 0.0024 18.6 -39.8 18.6 - 6.4 18.6
63 ✓ GPT-6 Luna (Medium)
OpenAI		 2026-09-22 $0.0422 - 0.0023 18.3 -40.1 18.3 - 6.3 18.3
64 Beam 1.0
Reflection		 2026-10-16 $0.284 1.01m 0.0160 17.7 -40.7 17.7 - 14.5 17.7
65 Nemotron 3.5 Super VL
NVIDIA		 2026-10-15 $0.557 2.64m 0.0409 13.6 -44.8 13.6 - 17.5 13.6
66 Inkling Small
Thinking Machines		 2026-07-30 $0.203 2.46m 0.0157 12.9 -45.5 12.9 - 13.1 12.9
67 Inkling (Xhigh)
Thinking Machines		 2026-07-15 $1.35 3.45m 0.1083 12.5 -45.9 12.5 - 21.3 12.5
68 Nemotron 3 Ultra 550B A55B (Reasoning)
NVIDIA		 2026-06-04 $2.23 4.23m 0.1893 11.8 -46.6 11.8 - 23.5 11.8
69 Gemini 3.5 Flash-Lite
Google		 2026-07-21 $0.693 1.08m 0.0594 11.7 -46.7 11.7 - 18.4 11.7
70 ✓ GPT-6 Luna (Low)
OpenAI		 2026-09-22 $0.00512 28s 0.0005 9.7 -48.7 9.7 - 0.0 9.7
71 GPT-6 Luna (Non-reasoning)
OpenAI		 2026-09-22 $0.0714 1.44m 0.0076 9.4 -49.0 9.4 - 8.5 9.4
72 Muse Glimmer (High)
Meta		 2026-08-10 $0.161 3.50m 0.0254 6.3 -52.1 6.3 - 12.1 6.3