Run #3297

success · fetched 2026-10-28 21:00:56 · 1.52 MB raw HTML · 73 models

Open this run in comparison view · JSON results

Top quality: Gemini 4 Carbon (High) (59.4 pts)

Cost/task vs quality?

73 models · 16 on the Pareto frontier · intelligence quality vs Cost per Task

318334863$0.01$0.10$1$10 cost per task, log scale intelligence quality Claude Opus 5.5 (Max, Default Fallback) · Cost/task: $10.0 · Time/task: 19.1m · Quality: 58.4GPT-6 Astra (Max) · Cost/task: $6.24 · Time/task: 11.6m · Quality: 56.9Claude Sonnet 5.5 (Max, Default Fallback) · Cost/task: $10.0 · Time/task: 22.1m · Quality: 54.2Claude Opus 5.5 (High, Default Fallback) · Cost/task: $3.46 · Time/task: 7.73m · Quality: 51.7GPT-6 Astra (Medium) · Cost/task: $2.73 · Time/task: 4.75m · Quality: 51.5GPT-6.1 Terra (Max) · Cost/task: $0.961 · Time/task: 10.9m · Quality: 51.0Gemini 4 Argon (High) · Cost/task: $7.68 · Time/task: 22.7m · Quality: 50.4Claude Fable 5.1 (Xhigh, Default Fallback) · Cost/task: $10.9 · Time/task: 15.8m · Quality: 49.7Claude Sonnet 5.5 (Xhigh, Default Fallback) · Cost/task: $4.09 · Time/task: 11.3m · Quality: 49.6Claude Fable 5.1 (Max, Default Fallback) · Cost/task: $13.0 · Time/task: 19.6m · Quality: 49.0Claude Opus 5.5 (Medium, Default Fallback) · Cost/task: $2.45 · Time/task: 5.52m · Quality: 48.1Claude Fable 5.1 (High, Default Fallback) · Cost/task: $7.56 · Time/task: 11.1m · Quality: 47.0Kimi K3.1 Pro · Cost/task: $3.68 · Time/task: 23.9m · Quality: 46.0GPT-6.1 Terra (Xhigh) · Cost/task: $0.401 · Time/task: 4.78m · Quality: 45.0Muse Wonder (Max) · Cost/task: $6.95 · Time/task: 7.60m · Quality: 43.3GPT-6 Astra (Low) · Cost/task: $1.42 · Time/task: 2.62m · Quality: 43.0Claude Fable 5.1 (Medium, Default Fallback) · Cost/task: $5.58 · Time/task: 8.00m · Quality: 43.0Claude Sonnet 5.5 (High, Default Fallback) · Cost/task: $1.64 · Time/task: 5.42m · Quality: 41.1Claude Fable 5.1 (Low, Default Fallback) · Cost/task: $4.49 · Time/task: 6.50m · Quality: 38.5Muse Spark 1.3 (Max) · Cost/task: $2.39 · Time/task: 7.80m · Quality: 36.4Claude Haiku 5.5 (Max) · Cost/task: $0.411 · Time/task: 9.90m · Quality: 36.2Grok 4.7 (High) · Cost/task: $6.18 · Time/task: 14.1m · Quality: 35.0Claude Opus 5.5 (Low, Default Fallback) · Cost/task: $0.976 · Time/task: 2.41m · Quality: 34.5Claude Haiku 5.5 (Xhigh) · Cost/task: $0.248 · Time/task: 7.57m · Quality: 34.3Grok 4.7 (Xhigh) · Cost/task: $7.51 · Time/task: 15.4m · Quality: 34.3Qwen3.8 Max (0902) · Cost/task: $10.0 · Time/task: 52.3m · Quality: 33.6DeepSeek V4.1 Flash 1017 (Max) · Cost/task: $0.602 · Time/task: 8.67m · Quality: 32.5GLM-5.3 (Max) · Cost/task: $5.01 · Time/task: 17.7m · Quality: 31.7Grok 4.7 (Low) · Cost/task: $2.92 · Time/task: 7.77m · Quality: 30.9Gemini 3.8 Flash (High) · Cost/task: $7.77 · Time/task: 11.4m · Quality: 30.5GPT-6.1 Terra (Non-reasoning) · Cost/task: $0.226 · Time/task: 1.73m · Quality: 30.4Claude Sonnet 5.5 (Medium, Default Fallback) · Cost/task: $0.781 · Time/task: 2.92m · Quality: 30.3Gemini 3.8 Flash (Medium) · Cost/task: $6.26 · Time/task: - · Quality: 30.0MiMo-V2.6-Pro · Cost/task: $0.257 · Time/task: 27.9m · Quality: 29.9Ling 3.1 Flash 1019 · Cost/task: $3.19 · Time/task: - · Quality: 29.5MiniMax-M3.1-Flash · Cost/task: $0.491 · Time/task: 17.2m · Quality: 28.9Step 5 Preview · Cost/task: $2.35 · Time/task: 9.67m · Quality: 28.5GPT-6 Luna (Max) · Cost/task: $0.152 · Time/task: 9.98m · Quality: 27.8MiMo-V2.6-Flash · Cost/task: $0.129 · Time/task: 33.4m · Quality: 26.7Claude Sonnet 5.5 (Low, Default Fallback) · Cost/task: $0.511 · Time/task: 1.98m · Quality: 25.3Mistral Large 4 Preview · Cost/task: $5.54 · Time/task: 19.1m · Quality: 25.3Muse Spark Open (Xhigh) · Cost/task: $2.02 · Time/task: 7.36m · Quality: 24.7Claude Haiku 5.5 (Medium) · Cost/task: $0.0918 · Time/task: 3.18m · Quality: 24.6GPT-6 Luna (Xhigh) · Cost/task: $0.104 · Time/task: 5.96m · Quality: 23.3DeepSeek V4 Pro 0813 (Max) · Cost/task: $1.77 · Time/task: 14.2m · Quality: 23.1GLM-5.3 (Low) · Cost/task: $1.57 · Time/task: 9.75m · Quality: 23.1GPT-6 Luna (High) · Cost/task: $0.0729 · Time/task: 4.46m · Quality: 21.4Qwen3.8 27B (Xhigh) · Cost/task: $3.53 · Time/task: 35.0m · Quality: 20.1MiniMax-M3 · Cost/task: $1.29 · Time/task: 11.7m · Quality: 18.7Beam 1.0 · Cost/task: $0.284 · Time/task: 1.13m · Quality: 17.7Nemotron 3.5 Super VL · Cost/task: $0.557 · Time/task: 2.66m · Quality: 13.6Inkling Small · Cost/task: $0.203 · Time/task: 2.50m · Quality: 12.9Inkling (Xhigh) · Cost/task: $1.35 · Time/task: 3.38m · Quality: 12.5Nemotron 3 Ultra 550B A55B (Reasoning) · Cost/task: $2.23 · Time/task: 4.50m · Quality: 11.8Gemini 3.5 Flash-Lite · Cost/task: $0.693 · Time/task: 1.04m · Quality: 11.7GPT-6 Luna (Non-reasoning) · Cost/task: $0.0714 · Time/task: 1.43m · Quality: 9.4Muse Glimmer (High) · Cost/task: $0.161 · Time/task: 3.51m · Quality: 6.3Gemini 4 Carbon (High) · Cost/task: $7.76 · Time/task: 20.8m · Quality: 59.4 · Pareto frontierClaude Opus 5.5 (Xhigh, Default Fallback) · Cost/task: $6.05 · Time/task: 13.0m · Quality: 57.0 · Pareto frontierGPT-6 Astra (Xhigh) · Cost/task: $3.66 · Time/task: 6.47m · Quality: 55.3 · Pareto frontierGPT-6 Astra (High) · Cost/task: $3.24 · Time/task: 6.21m · Quality: 54.8 · Pareto frontierGPT-6.1 Sol (Max) · Cost/task: $1.27 · Time/task: 13.9m · Quality: 54.5 · Pareto frontierGPT-6.1 Sol (Xhigh) · Cost/task: $0.764 · Time/task: 7.22m · Quality: 52.3 · Pareto frontierGPT-6.1 Sol (High) · Cost/task: $0.591 · Time/task: 5.27m · Quality: 50.2 · Pareto frontierGPT-6.1 Sol (Medium) · Cost/task: $0.359 · Time/task: 3.46m · Quality: 46.1 · Pareto frontierGPT-6.1 Terra (High) · Cost/task: $0.259 · Time/task: 3.04m · Quality: 42.3 · Pareto frontierGPT-6.1 Terra (Medium) · Cost/task: $0.169 · Time/task: 1.98m · Quality: 38.1 · Pareto frontierGPT-6.1 Sol (Low) · Cost/task: $0.161 · Time/task: 1.55m · Quality: 34.5 · Pareto frontierClaude Haiku 5.5 (High) · Cost/task: $0.159 · Time/task: 4.87m · Quality: 29.6 · Pareto frontierGPT-6.1 Terra (Low) · Cost/task: $0.0713 · Time/task: 48s · Quality: 29.2 · Pareto frontierClaude Haiku 5.5 (Low) · Cost/task: $0.0438 · Time/task: 1.63m · Quality: 18.6 · Pareto frontierGPT-6 Luna (Medium) · Cost/task: $0.0422 · Time/task: - · Quality: 18.3 · Pareto frontierGPT-6 Luna (Low) · Cost/task: $0.00512 · Time/task: 27s · Quality: 9.7 · Pareto frontier GPT-6 Luna (low)GPT-6 Luna (mediu…Claude Haiku 5.5 …GPT-6.1 Terra (lo…Claude Haiku 5.5 …GPT-6.1 Sol (low)GPT-6.1 Terra (me…GPT-6.1 Terra (hi…GPT-6.1 Sol (medi…GPT-6.1 Sol (high)GPT-6.1 Sol (xhig…GPT-6.1 Sol (max)GPT-6 Astra (high)GPT-6 Astra (xhig…Claude Opus 5.5 (…Gemini 4 Carbon (…
#?Pareto?Model?Released?Cost/task?Time/task?$/Q?Qual?ΔTop?Intel?Agent?Pen?Score?
1 ✓ Gemini 4 Carbon (High)
Google		 2026-10-28 $7.76 20.8m 0.1305 59.4 0.0 59.4 - 28.9 59.4
2 Claude Opus 5.5 (Max, Default Fallback)
Anthropic		 2026-09-22 $10.0 19.1m 0.1718 58.4 -1.0 58.4 - 30.0 58.4
3 ✓ Claude Opus 5.5 (Xhigh, Default Fallback)
Anthropic		 2026-09-22 $6.05 13.0m 0.1061 57.0 -2.4 57.0 - 27.8 57.0
4 GPT-6 Astra (Max)
OpenAI		 2026-09-03 $6.24 11.6m 0.1098 56.9 -2.6 56.9 - 28.0 56.9
5 ✓ GPT-6 Astra (Xhigh)
OpenAI		 2026-09-03 $3.66 6.47m 0.0662 55.3 -4.2 55.3 - 25.6 55.3
6 ✓ GPT-6 Astra (High)
OpenAI		 2026-09-03 $3.24 6.21m 0.0590 54.8 -4.6 54.8 - 25.1 54.8
7 ✓ GPT-6.1 Sol (Max)
OpenAI		 2026-09-29 $1.27 13.9m 0.0233 54.5 -4.9 54.5 - 21.0 54.5
8 Claude Sonnet 5.5 (Max, Default Fallback)
Anthropic		 2026-09-28 $10.0 22.1m 0.1847 54.2 -5.2 54.2 - 30.0 54.2
9 ✓ GPT-6.1 Sol (Xhigh)
OpenAI		 2026-09-29 $0.764 7.22m 0.0146 52.3 -7.1 52.3 - 18.8 52.3
10 Claude Opus 5.5 (High, Default Fallback)
Anthropic		 2026-09-22 $3.46 7.73m 0.0670 51.7 -7.7 51.7 - 25.4 51.7
11 GPT-6 Astra (Medium)
OpenAI		 2026-09-03 $2.73 4.75m 0.0531 51.5 -8.0 51.5 - 24.4 51.5
12 GPT-6.1 Terra (Max)
OpenAI		 2026-10-26 $0.961 10.9m 0.0189 51.0 -8.4 51.0 - 19.8 51.0
13 Gemini 4 Argon (High)
Google		 2026-09-30 $7.68 22.7m 0.1524 50.4 -9.0 50.4 - 28.9 50.4
14 ✓ GPT-6.1 Sol (High)
OpenAI		 2026-09-29 $0.591 5.27m 0.0118 50.2 -9.2 50.2 - 17.7 50.2
15 Claude Fable 5.1 (Xhigh, Default Fallback)
Anthropic		 2026-09-01 $10.9 15.8m 0.2191 49.7 -9.8 49.7 - 30.4 49.7
16 Claude Sonnet 5.5 (Xhigh, Default Fallback)
Anthropic		 2026-09-28 $4.09 11.3m 0.0825 49.6 -9.9 49.6 - 26.1 49.6
17 Claude Fable 5.1 (Max, Default Fallback)
Anthropic		 2026-09-01 $13.0 19.6m 0.2661 49.0 -10.4 49.0 - 31.2 49.0
18 Claude Opus 5.5 (Medium, Default Fallback)
Anthropic		 2026-09-22 $2.45 5.52m 0.0510 48.1 -11.4 48.1 - 23.9 48.1
19 Claude Fable 5.1 (High, Default Fallback)
Anthropic		 2026-09-01 $7.56 11.1m 0.1608 47.0 -12.4 47.0 - 28.8 47.0
20 ✓ GPT-6.1 Sol (Medium)
OpenAI		 2026-09-29 $0.359 3.46m 0.0078 46.1 -13.3 46.1 - 15.6 46.1
21 Kimi K3.1 Pro
Kimi		 2026-10-23 $3.68 23.9m 0.0800 46.0 -13.4 46.0 - 25.7 46.0
22 GPT-6.1 Terra (Xhigh)
OpenAI		 2026-10-26 $0.401 4.78m 0.0089 45.0 -14.5 45.0 - 16.0 45.0
23 Muse Wonder (Max)
Meta		 2026-10-12 $6.95 7.60m 0.1605 43.3 -16.1 43.3 - 28.4 43.3
24 GPT-6 Astra (Low)
OpenAI		 2026-09-03 $1.42 2.62m 0.0331 43.0 -16.4 43.0 - 21.5 43.0
25 Claude Fable 5.1 (Medium, Default Fallback)
Anthropic		 2026-09-01 $5.58 8.00m 0.1299 43.0 -16.5 43.0 - 27.5 43.0
26 ✓ GPT-6.1 Terra (High)
OpenAI		 2026-10-26 $0.259 3.04m 0.0061 42.3 -17.1 42.3 - 14.1 42.3
27 Claude Sonnet 5.5 (High, Default Fallback)
Anthropic		 2026-09-28 $1.64 5.42m 0.0398 41.1 -18.3 41.1 - 22.1 41.1
28 Claude Fable 5.1 (Low, Default Fallback)
Anthropic		 2026-09-01 $4.49 6.50m 0.1166 38.5 -20.9 38.5 - 26.5 38.5
29 ✓ GPT-6.1 Terra (Medium)
OpenAI		 2026-10-26 $0.169 1.98m 0.0044 38.1 -21.3 38.1 - 12.3 38.1
30 Muse Spark 1.3 (Max)
Meta		 2026-09-02 $2.39 7.80m 0.0655 36.4 -23.0 36.4 - 23.8 36.4
31 Claude Haiku 5.5 (Max)
Anthropic		 2026-10-07 $0.411 9.90m 0.0113 36.2 -23.2 36.2 - 16.1 36.2
32 Grok 4.7 (High)
SpaceXAI		 2026-09-21 $6.18 14.1m 0.1766 35.0 -24.4 35.0 - 27.9 35.0
33 ✓ GPT-6.1 Sol (Low)
OpenAI		 2026-09-29 $0.161 1.55m 0.0047 34.5 -24.9 34.5 - 12.1 34.5
34 Claude Opus 5.5 (Low, Default Fallback)
Anthropic		 2026-09-22 $0.976 2.41m 0.0283 34.5 -24.9 34.5 - 19.9 34.5
35 Claude Haiku 5.5 (Xhigh)
Anthropic		 2026-10-07 $0.248 7.57m 0.0072 34.3 -25.1 34.3 - 14.0 34.3
36 Grok 4.7 (Xhigh)
SpaceXAI		 2026-09-21 $7.51 15.4m 0.2188 34.3 -25.1 34.3 - 28.8 34.3
37 Qwen3.8 Max (0902)
Alibaba		 2026-09-02 $10.0 52.3m 0.2987 33.6 -25.9 33.6 - 30.0 33.6
38 DeepSeek V4.1 Flash 1017 (Max)
DeepSeek		 2026-10-17 $0.602 8.67m 0.0185 32.5 -26.9 32.5 - 17.8 32.5
39 GLM-5.3 (Max)
Z AI		 2026-08-18 $5.01 17.7m 0.1583 31.7 -27.8 31.7 - 27.0 31.7
40 Grok 4.7 (Low)
SpaceXAI		 2026-09-21 $2.92 7.77m 0.0946 30.9 -28.5 30.9 - 24.7 30.9
41 Gemini 3.8 Flash (High)
Google		 2026-09-02 $7.77 11.4m 0.2542 30.5 -28.9 30.5 - 28.9 30.5
42 GPT-6.1 Terra (Non-reasoning)
OpenAI		 2026-10-26 $0.226 1.73m 0.0074 30.4 -29.0 30.4 - 13.5 30.4
43 Claude Sonnet 5.5 (Medium, Default Fallback)
Anthropic		 2026-09-28 $0.781 2.92m 0.0257 30.3 -29.1 30.3 - 18.9 30.3
44 Gemini 3.8 Flash (Medium)
Google		 2026-09-02 $6.26 - 0.2088 30.0 -29.4 30.0 - 28.0 30.0
45 MiMo-V2.6-Pro
Xiaomi		 2026-09-21 $0.257 27.9m 0.0086 29.9 -29.5 29.9 - 14.1 29.9
46 ✓ Claude Haiku 5.5 (High)
Anthropic		 2026-10-07 $0.159 4.87m 0.0054 29.6 -29.8 29.6 - 12.0 29.6
47 Ling 3.1 Flash 1019
InclusionAI		 2026-10-22 $3.19 - 0.1081 29.5 -30.0 29.5 - 25.0 29.5
48 ✓ GPT-6.1 Terra (Low)
OpenAI		 2026-10-26 $0.0713 48s 0.0024 29.2 -30.2 29.2 - 8.5 29.2
49 MiniMax-M3.1-Flash
MiniMax		 2026-10-19 $0.491 17.2m 0.0170 28.9 -30.5 28.9 - 16.9 28.9
50 Step 5 Preview
StepFun		 2026-09-18 $2.35 9.67m 0.0822 28.5 -30.9 28.5 - 23.7 28.5
51 GPT-6 Luna (Max)
OpenAI		 2026-09-22 $0.152 9.98m 0.0055 27.8 -31.7 27.8 - 11.8 27.8
52 MiMo-V2.6-Flash
Xiaomi		 2026-09-21 $0.129 33.4m 0.0048 26.7 -32.7 26.7 - 11.1 26.7
53 Claude Sonnet 5.5 (Low, Default Fallback)
Anthropic		 2026-09-28 $0.511 1.98m 0.0202 25.3 -34.1 25.3 - 17.1 25.3
54 Mistral Large 4 Preview
Mistral		 2026-10-06 $5.54 19.1m 0.2190 25.3 -34.1 25.3 - 27.4 25.3
55 Muse Spark Open (Xhigh)
Meta		 2026-10-15 $2.02 7.36m 0.0817 24.7 -34.7 24.7 - 23.0 24.7
56 Claude Haiku 5.5 (Medium)
Anthropic		 2026-10-07 $0.0918 3.18m 0.0037 24.6 -34.9 24.6 - 9.6 24.6
57 GPT-6 Luna (Xhigh)
OpenAI		 2026-09-22 $0.104 5.96m 0.0045 23.3 -36.1 23.3 - 10.2 23.3
58 DeepSeek V4 Pro 0813 (Max)
DeepSeek		 2026-08-13 $1.77 14.2m 0.0767 23.1 -36.4 23.1 - 22.5 23.1
59 GLM-5.3 (Low)
Z AI		 2026-08-18 $1.57 9.75m 0.0683 23.1 -36.4 23.1 - 22.0 23.1
60 GPT-6 Luna (High)
OpenAI		 2026-09-22 $0.0729 4.46m 0.0034 21.4 -38.0 21.4 - 8.6 21.4
61 Qwen3.8 27B (Xhigh)
Alibaba		 2026-08-14 $3.53 35.0m 0.1755 20.1 -39.3 20.1 - 25.5 20.1
62 MiniMax-M3
MiniMax		 2026-06-01 $1.29 11.7m 0.0686 18.7 -40.7 18.7 - 21.1 18.7
63 ✓ Claude Haiku 5.5 (Low)
Anthropic		 2026-10-07 $0.0438 1.63m 0.0024 18.6 -40.8 18.6 - 6.4 18.6
64 ✓ GPT-6 Luna (Medium)
OpenAI		 2026-09-22 $0.0422 - 0.0023 18.3 -41.1 18.3 - 6.3 18.3
65 Beam 1.0
Reflection		 2026-10-16 $0.284 1.13m 0.0160 17.7 -41.7 17.7 - 14.5 17.7
66 Nemotron 3.5 Super VL
NVIDIA		 2026-10-15 $0.557 2.66m 0.0409 13.6 -45.8 13.6 - 17.5 13.6
67 Inkling Small
Thinking Machines		 2026-07-30 $0.203 2.50m 0.0157 12.9 -46.5 12.9 - 13.1 12.9
68 Inkling (Xhigh)
Thinking Machines		 2026-07-15 $1.35 3.38m 0.1083 12.5 -46.9 12.5 - 21.3 12.5
69 Nemotron 3 Ultra 550B A55B (Reasoning)
NVIDIA		 2026-06-04 $2.23 4.50m 0.1893 11.8 -47.7 11.8 - 23.5 11.8
70 Gemini 3.5 Flash-Lite
Google		 2026-07-21 $0.693 1.04m 0.0594 11.7 -47.8 11.7 - 18.4 11.7
71 ✓ GPT-6 Luna (Low)
OpenAI		 2026-09-22 $0.00512 27s 0.0005 9.7 -49.8 9.7 - 0.0 9.7
72 GPT-6 Luna (Non-reasoning)
OpenAI		 2026-09-22 $0.0714 1.43m 0.0076 9.4 -50.0 9.4 - 8.5 9.4
73 Muse Glimmer (High)
Meta		 2026-08-10 $0.161 3.51m 0.0254 6.3 -53.1 6.3 - 12.1 6.3