Run #2077
success · fetched 2026-09-06 13:00:21 · 1.31 MB raw HTML · 53 models
Open this run in comparison view · Download raw HTML · JSON results
Top quality: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (56.8 pts)
Cost/task vs quality?
53 models · 16 on the Pareto frontier · intelligence quality vs Cost per Task
| #? | Pareto? | Model? | Released? | Cost/task? | Time/task? | $/Q? | Qual? | ΔTop? | Intel? | Agent? | Pen? | Score? |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ✓ | Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) Anthropic |
2026-09-01 | $6.12 | 9.53m | 0.1078 | 56.8 | 0.0 | 56.8 | - | 27.9 | 56.8 |
| 2 | ✓ | GPT-6 Astra (max) OpenAI |
2026-09-03 | $2.57 | 4.93m | 0.0470 | 54.7 | -2.1 | 54.7 | - | 24.1 | 54.7 |
| 3 | ✓ | GPT-6 Astra (xhigh) OpenAI |
2026-09-03 | $1.85 | 3.30m | 0.0340 | 54.3 | -2.4 | 54.3 | - | 22.7 | 54.3 |
| 4 | Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic |
2026-07-24 | $4.21 | 10.0m | 0.0778 | 54.1 | -2.7 | 54.1 | - | 26.2 | 54.1 | |
| 5 | Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic |
2026-07-24 | $3.36 | 8.38m | 0.0629 | 53.4 | -3.4 | 53.4 | - | 25.3 | 53.4 | |
| 6 | ✓ | GPT-6 Astra (high) OpenAI |
2026-09-03 | $1.41 | 2.33m | 0.0264 | 53.4 | -3.4 | 53.4 | - | 21.5 | 53.4 |
| 7 | Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic |
2026-06-09 | $5.62 | 7.48m | 0.1056 | 53.2 | -3.6 | 53.2 | - | 27.5 | 53.2 | |
| 8 | ✓ | Muse Spark 1.3 (max) Meta |
2026-09-02 | $0.959 | 3.59m | 0.0181 | 53.0 | -3.8 | 53.0 | - | 19.8 | 53.0 |
| 9 | GPT-6 Astra (medium) OpenAI |
2026-09-03 | $1.16 | 1.87m | 0.0222 | 52.2 | -4.5 | 52.2 | - | 20.6 | 52.2 | |
| 10 | Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic |
2026-07-24 | $2.44 | 6.60m | 0.0470 | 52.0 | -4.7 | 52.0 | - | 23.9 | 52.0 | |
| 11 | ✓ | Muse Spark 1.3 (xhigh) Meta |
2026-09-02 | $0.840 | 3.34m | 0.0163 | 51.6 | -5.2 | 51.6 | - | 19.2 | 51.6 |
| 12 | GPT-5.6 Sol (max) OpenAI |
2026-07-09 | $1.25 | 4.53m | 0.0244 | 51.3 | -5.5 | 51.3 | - | 21.0 | 51.3 | |
| 13 | Grok 4.6 (high) SpaceXAI |
2026-08-12 | $1.25 | 7.53m | 0.0248 | 50.6 | -6.2 | 50.6 | - | 21.0 | 50.6 | |
| 14 | Kimi K3 (max) Kimi |
2026-07-16 | $1.58 | 13.9m | 0.0314 | 50.2 | -6.5 | 50.2 | - | 22.0 | 50.2 | |
| 15 | GPT-5.6 Sol (xhigh) OpenAI |
2026-07-09 | $0.889 | 3.16m | 0.0178 | 49.8 | -6.9 | 49.8 | - | 19.5 | 49.8 | |
| 16 | Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic |
2026-07-24 | $1.36 | 4.00m | 0.0275 | 49.5 | -7.2 | 49.5 | - | 21.3 | 49.5 | |
| 17 | ✓ | GPT-6 Astra (low) OpenAI |
2026-09-03 | $0.631 | 50s | 0.0128 | 49.3 | -7.4 | 49.3 | - | 18.0 | 49.3 |
| 18 | GLM-5.3 (max) Z AI |
2026-08-18 | $1.26 | 7.34m | 0.0259 | 48.6 | -8.2 | 48.6 | - | 21.0 | 48.6 | |
| 19 | ✓ | GPT-5.6 Sol (high) OpenAI |
2026-07-09 | $0.615 | 2.40m | 0.0127 | 48.3 | -8.5 | 48.3 | - | 17.9 | 48.3 |
| 20 | GPT-6 Astra (Non-reasoning) OpenAI |
2026-09-03 | $1.42 | - | 0.0297 | 47.8 | -8.9 | 47.8 | - | 21.5 | 47.8 | |
| 21 | Gemini 3.8 Flash (high) |
2026-09-02 | $0.738 | 2.74m | 0.0157 | 47.1 | -9.7 | 47.1 | - | 18.7 | 47.1 | |
| 22 | Qwen3.8 Max Alibaba |
2026-08-03 | $1.19 | 13.3m | 0.0253 | 46.9 | -9.9 | 46.9 | - | 20.7 | 46.9 | |
| 23 | ✓ | Muse Spark 1.2 (xhigh) Meta |
2026-08-05 | $0.545 | 2.08m | 0.0116 | 46.8 | -9.9 | 46.8 | - | 17.4 | 46.8 |
| 24 | GPT-5.6 Terra (max) OpenAI |
2026-07-09 | $0.814 | 3.93m | 0.0174 | 46.8 | -10.0 | 46.8 | - | 19.1 | 46.8 | |
| 25 | Qwen3.8 2.4T A95B Alibaba |
2026-08-12 | $1.10 | 16.1m | 0.0235 | 46.7 | -10.0 | 46.7 | - | 20.4 | 46.7 | |
| 26 | ✓ | GLM-5.3-Flash Z AI |
2026-08-26 | $0.183 | 13.8m | 0.0039 | 46.2 | -10.5 | 46.2 | - | 12.6 | 46.2 |
| 27 | GPT-5.6 Sol (medium) OpenAI |
2026-07-09 | $0.367 | 1.32m | 0.0080 | 46.0 | -10.8 | 46.0 | - | 15.6 | 46.0 | |
| 28 | Grok 4.5 (high) SpaceXAI |
2026-07-08 | $0.434 | 5.81m | 0.0095 | 45.5 | -11.3 | 45.5 | - | 16.4 | 45.5 | |
| 29 | Gemini 3.7 Flash (high) |
2026-08-13 | $0.548 | 2.31m | 0.0121 | 45.2 | -11.5 | 45.2 | - | 17.4 | 45.2 | |
| 30 | Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic |
2026-06-30 | $3.31 | 10.7m | 0.0735 | 45.1 | -11.6 | 45.1 | - | 25.2 | 45.1 | |
| 31 | GPT-5.6 Terra (xhigh) OpenAI |
2026-07-09 | $0.502 | 2.80m | 0.0113 | 44.4 | -12.4 | 44.4 | - | 17.0 | 44.4 | |
| 32 | Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic |
2026-07-24 | $0.636 | 1.91m | 0.0145 | 43.8 | -13.0 | 43.8 | - | 18.0 | 43.8 | |
| 33 | ✓ | GPT-5.6 Luna (max) OpenAI |
2026-07-09 | $0.0969 | 3.89m | 0.0022 | 43.4 | -13.3 | 43.4 | - | 9.9 | 43.4 |
| 34 | DeepSeek V4 Pro 0813 (Reasoning, Max Effort) DeepSeek |
2026-08-13 | $0.333 | 7.51m | 0.0079 | 42.1 | -14.6 | 42.1 | - | 15.2 | 42.1 | |
| 35 | ✓ | GPT-5.6 Luna (xhigh) OpenAI |
2026-07-09 | $0.0574 | 2.61m | 0.0014 | 41.6 | -15.2 | 41.6 | - | 7.6 | 41.6 |
| 36 | Qwen3.8 27B (xhigh) Alibaba |
2026-08-14 | $0.573 | 14.6m | 0.0138 | 41.4 | -15.4 | 41.4 | - | 17.6 | 41.4 | |
| 37 | GPT-5.6 Terra (high) OpenAI |
2026-07-09 | $0.298 | 1.73m | 0.0072 | 41.3 | -15.5 | 41.3 | - | 14.7 | 41.3 | |
| 38 | GPT-5.6 Sol (low) OpenAI |
2026-07-09 | $0.225 | 43s | 0.0055 | 40.8 | -15.9 | 40.8 | - | 13.5 | 40.8 | |
| 39 | DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek |
2026-07-31 | $0.140 | 4.83m | 0.0034 | 40.8 | -15.9 | 40.8 | - | 11.5 | 40.8 | |
| 40 | Gemini 3.6 Flash (high) |
2026-07-21 | $0.356 | 2.21m | 0.0088 | 40.3 | -16.5 | 40.3 | - | 15.5 | 40.3 | |
| 41 | Gemini 3.5 Flash (high) |
2026-05-19 | $1.01 | 3.21m | 0.0255 | 39.7 | -17.0 | 39.7 | - | 20.1 | 39.7 | |
| 42 | Gemini 3.1 Pro Preview |
2026-02-19 | $0.335 | 1.94m | 0.0091 | 36.7 | -20.1 | 36.7 | - | 15.3 | 36.7 | |
| 43 | MiniMax-M3 MiniMax |
2026-06-01 | $0.229 | 6.00m | 0.0064 | 35.7 | -21.0 | 35.7 | - | 13.6 | 35.7 | |
| 44 | ✓ | MiMo-V2.5-Pro Xiaomi |
2026-04-22 | $0.0368 | 9.55m | 0.0011 | 32.6 | -24.2 | 32.6 | - | 5.7 | 32.6 |
| 45 | Inkling (xhigh) Thinking Machines |
2026-07-15 | $0.438 | 7.21m | 0.0136 | 32.2 | -24.6 | 32.2 | - | 16.4 | 32.2 | |
| 46 | Gemini 3.5 Flash-Lite |
2026-07-21 | $0.0796 | 37s | 0.0029 | 27.6 | -29.2 | 27.6 | - | 9.0 | 27.6 | |
| 47 | ✓ | Ling 3.0 Flash InclusionAI |
2026-08-04 | $0.0306 | 1.61m | 0.0011 | 27.4 | -29.4 | 27.4 | - | 4.9 | 27.4 |
| 48 | Muse Glimmer (high) Meta |
2026-08-10 | $0.0429 | 1.99m | 0.0018 | 24.4 | -32.4 | 24.4 | - | 6.3 | 24.4 | |
| 49 | Claude 4.5 Haiku (Reasoning) Anthropic |
2025-10-15 | $0.199 | 2.94m | 0.0089 | 22.5 | -34.3 | 22.5 | - | 13.0 | 22.5 | |
| 50 | gpt-oss-120b (high) OpenAI |
2025-08-05 | $0.123 | 3.10m | 0.0079 | 15.6 | -41.2 | 15.6 | - | 10.9 | 15.6 | |
| 51 | ✓ | Granite 4.2 8B IBM |
2026-08-25 | $0.0230 | 3.56m | 0.0017 | 13.9 | -42.8 | 13.9 | - | 3.6 | 13.9 |
| 52 | Mistral Large 3 Mistral |
2025-12-02 | $0.0689 | 46s | 0.0062 | 11.1 | -45.6 | 11.1 | - | 8.4 | 11.1 | |
| 53 | ✓ | Granite 4.2 3B IBM |
2026-08-25 | $0.00552 | 47s | 0.0005 | 10.7 | -46.1 | 10.7 | - | 0.0 | 10.7 |