Run #2088
success · fetched 2026-09-07 00:00:19 · 1.31 MB raw HTML · 75 models
Open this run in comparison view · Download raw HTML · JSON results
Top quality: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (56.8 pts)
Cost/task vs quality?
75 models · 18 on the Pareto frontier · intelligence quality vs Cost per Task
| #? | Pareto? | Model? | Released? | Cost/task? | Time/task? | $/Q? | Qual? | ΔTop? | Intel? | Agent? | Pen? | Score? |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ✓ | Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) Anthropic |
2026-09-01 | $6.12 | 9.88m | 0.1078 | 56.8 | 0.0 | 56.8 | - | 27.9 | 56.8 |
| 2 | ✓ | Claude Fable 5.1 (Adaptive Reasoning, Xhigh Effort, Default Fallback) Anthropic |
2026-09-01 | $4.64 | 7.96m | 0.0825 | 56.2 | -0.5 | 56.2 | - | 26.7 | 56.2 |
| 3 | ✓ | GPT-6 Astra (max) OpenAI |
2026-09-03 | $2.57 | 5.17m | 0.0470 | 54.7 | -2.1 | 54.7 | - | 24.1 | 54.7 |
| 4 | Claude Fable 5.1 (Adaptive Reasoning, High Effort, Default Fallback) Anthropic |
2026-09-01 | $2.86 | 5.04m | 0.0525 | 54.5 | -2.3 | 54.5 | - | 24.6 | 54.5 | |
| 5 | ✓ | GPT-6 Astra (xhigh) OpenAI |
2026-09-03 | $1.85 | 3.56m | 0.0340 | 54.3 | -2.4 | 54.3 | - | 22.7 | 54.3 |
| 6 | Claude Opus 5 (Adaptive Reasoning, Max Effort) Anthropic |
2026-07-24 | $4.21 | 10.3m | 0.0778 | 54.1 | -2.7 | 54.1 | - | 26.2 | 54.1 | |
| 7 | Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) Anthropic |
2026-07-24 | $3.36 | 8.67m | 0.0629 | 53.4 | -3.4 | 53.4 | - | 25.3 | 53.4 | |
| 8 | ✓ | GPT-6 Astra (high) OpenAI |
2026-09-03 | $1.41 | 2.46m | 0.0264 | 53.4 | -3.4 | 53.4 | - | 21.5 | 53.4 |
| 9 | Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic |
2026-06-09 | $5.62 | 7.79m | 0.1056 | 53.2 | -3.6 | 53.2 | - | 27.5 | 53.2 | |
| 10 | ✓ | Muse Spark 1.3 (max) Meta |
2026-09-02 | $0.959 | 3.12m | 0.0181 | 53.0 | -3.8 | 53.0 | - | 19.8 | 53.0 |
| 11 | Claude Fable 5.1 (Adaptive Reasoning, Medium Effort, Default Fallback) Anthropic |
2026-09-01 | $2.14 | 3.67m | 0.0405 | 52.8 | -4.0 | 52.8 | - | 23.3 | 52.8 | |
| 12 | GPT-6 Astra (medium) OpenAI |
2026-09-03 | $1.16 | 1.93m | 0.0222 | 52.2 | -4.5 | 52.2 | - | 20.6 | 52.2 | |
| 13 | Claude Opus 5 (Adaptive Reasoning, High Effort) Anthropic |
2026-07-24 | $2.44 | 6.71m | 0.0470 | 52.0 | -4.7 | 52.0 | - | 23.9 | 52.0 | |
| 14 | ✓ | Muse Spark 1.3 (xhigh) Meta |
2026-09-02 | $0.840 | 3.33m | 0.0163 | 51.6 | -5.2 | 51.6 | - | 19.2 | 51.6 |
| 15 | GPT-5.6 Sol (max) OpenAI |
2026-07-09 | $1.25 | 4.99m | 0.0244 | 51.3 | -5.5 | 51.3 | - | 21.0 | 51.3 | |
| 16 | Claude Fable 5.1 (Adaptive Reasoning, Low Effort, Default Fallback) Anthropic |
2026-09-01 | $1.70 | 2.86m | 0.0334 | 50.9 | -5.9 | 50.9 | - | 22.3 | 50.9 | |
| 17 | Grok 4.6 (high) SpaceXAI |
2026-08-12 | $1.25 | 7.80m | 0.0248 | 50.6 | -6.2 | 50.6 | - | 21.0 | 50.6 | |
| 18 | Kimi K3 (max) Kimi |
2026-07-16 | $1.58 | 14.2m | 0.0314 | 50.2 | -6.5 | 50.2 | - | 22.0 | 50.2 | |
| 19 | Grok 4.6 (xhigh) SpaceXAI |
2026-08-12 | $1.61 | 8.49m | 0.0323 | 49.9 | -6.8 | 49.9 | - | 22.1 | 49.9 | |
| 20 | GPT-5.6 Sol (xhigh) OpenAI |
2026-07-09 | $0.889 | 3.30m | 0.0178 | 49.8 | -6.9 | 49.8 | - | 19.5 | 49.8 | |
| 21 | Claude Opus 5 (Adaptive Reasoning, Medium Effort) Anthropic |
2026-07-24 | $1.36 | 4.01m | 0.0275 | 49.5 | -7.2 | 49.5 | - | 21.3 | 49.5 | |
| 22 | ✓ | GPT-6 Astra (low) OpenAI |
2026-09-03 | $0.631 | 51s | 0.0128 | 49.3 | -7.4 | 49.3 | - | 18.0 | 49.3 |
| 23 | Grok 4.6 (medium) SpaceXAI |
2026-08-12 | $0.947 | 6.19m | 0.0194 | 48.8 | -7.9 | 48.8 | - | 19.8 | 48.8 | |
| 24 | GLM-5.3 (max) Z AI |
2026-08-18 | $1.26 | 7.42m | 0.0259 | 48.6 | -8.2 | 48.6 | - | 21.0 | 48.6 | |
| 25 | ✓ | GPT-5.6 Sol (high) OpenAI |
2026-07-09 | $0.615 | 2.41m | 0.0127 | 48.3 | -8.5 | 48.3 | - | 17.9 | 48.3 |
| 26 | GPT-6 Astra (Non-reasoning) OpenAI |
2026-09-03 | $1.42 | - | 0.0297 | 47.8 | -8.9 | 47.8 | - | 21.5 | 47.8 | |
| 27 | Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic |
2026-05-28 | $2.60 | 8.65m | 0.0543 | 47.8 | -9.0 | 47.8 | - | 24.1 | 47.8 | |
| 28 | Gemini 3.8 Flash (high) |
2026-09-02 | $0.738 | 3.09m | 0.0157 | 47.1 | -9.7 | 47.1 | - | 18.7 | 47.1 | |
| 29 | Qwen3.8 Max Alibaba |
2026-08-03 | $1.19 | 13.9m | 0.0253 | 46.9 | -9.9 | 46.9 | - | 20.7 | 46.9 | |
| 30 | ✓ | Muse Spark 1.2 (xhigh) Meta |
2026-08-05 | $0.545 | 2.28m | 0.0116 | 46.8 | -9.9 | 46.8 | - | 17.4 | 46.8 |
| 31 | GPT-5.6 Terra (max) OpenAI |
2026-07-09 | $0.814 | 3.95m | 0.0174 | 46.8 | -10.0 | 46.8 | - | 19.1 | 46.8 | |
| 32 | Qwen3.8 2.4T A95B Alibaba |
2026-08-12 | $1.10 | 16.2m | 0.0235 | 46.7 | -10.0 | 46.7 | - | 20.4 | 46.7 | |
| 33 | ✓ | GLM-5.3-Flash Z AI |
2026-08-26 | $0.183 | 13.7m | 0.0039 | 46.2 | -10.5 | 46.2 | - | 12.6 | 46.2 |
| 34 | GPT-5.6 Sol (medium) OpenAI |
2026-07-09 | $0.367 | 1.36m | 0.0080 | 46.0 | -10.8 | 46.0 | - | 15.6 | 46.0 | |
| 35 | Grok 4.5 (high) SpaceXAI |
2026-07-08 | $0.434 | 5.89m | 0.0095 | 45.5 | -11.3 | 45.5 | - | 16.4 | 45.5 | |
| 36 | Gemini 3.7 Flash (high) |
2026-08-13 | $0.548 | 2.48m | 0.0121 | 45.2 | -11.5 | 45.2 | - | 17.4 | 45.2 | |
| 37 | Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic |
2026-06-30 | $3.31 | 12.1m | 0.0735 | 45.1 | -11.6 | 45.1 | - | 25.2 | 45.1 | |
| 38 | GPT-5.6 Terra (xhigh) OpenAI |
2026-07-09 | $0.502 | 2.76m | 0.0113 | 44.4 | -12.4 | 44.4 | - | 17.0 | 44.4 | |
| 39 | Claude Opus 5 (Adaptive Reasoning, Low Effort) Anthropic |
2026-07-24 | $0.636 | 1.91m | 0.0145 | 43.8 | -13.0 | 43.8 | - | 18.0 | 43.8 | |
| 40 | ✓ | GPT-5.6 Luna (max) OpenAI |
2026-07-09 | $0.0969 | 3.89m | 0.0022 | 43.4 | -13.3 | 43.4 | - | 9.9 | 43.4 |
| 41 | DeepSeek V4 Pro 0813 (Reasoning, Max Effort) DeepSeek |
2026-08-13 | $0.333 | 7.51m | 0.0079 | 42.1 | -14.6 | 42.1 | - | 15.2 | 42.1 | |
| 42 | GLM-5.2 (max) Z AI |
2026-06-16 | $0.561 | 10.2m | 0.0133 | 42.1 | -14.7 | 42.1 | - | 17.5 | 42.1 | |
| 43 | Grok 4.6 (low) SpaceXAI |
2026-08-12 | $0.258 | 2.43m | 0.0062 | 41.6 | -15.2 | 41.6 | - | 14.1 | 41.6 | |
| 44 | ✓ | GPT-5.6 Luna (xhigh) OpenAI |
2026-07-09 | $0.0574 | 2.69m | 0.0014 | 41.6 | -15.2 | 41.6 | - | 7.6 | 41.6 |
| 45 | Qwen3.8 27B (xhigh) Alibaba |
2026-08-14 | $0.573 | 14.8m | 0.0138 | 41.4 | -15.4 | 41.4 | - | 17.6 | 41.4 | |
| 46 | GPT-5.6 Terra (high) OpenAI |
2026-07-09 | $0.298 | 1.73m | 0.0072 | 41.3 | -15.5 | 41.3 | - | 14.7 | 41.3 | |
| 47 | GPT-5.5 (medium) OpenAI |
2026-04-23 | $0.658 | 1.39m | 0.0160 | 41.2 | -15.5 | 41.2 | - | 18.2 | 41.2 | |
| 48 | Muse Spark 1.1 (xhigh) Meta |
2026-07-09 | $0.582 | - | 0.0141 | 41.2 | -15.6 | 41.2 | - | 17.6 | 41.2 | |
| 49 | GPT-5.6 Sol (low) OpenAI |
2026-07-09 | $0.225 | 45s | 0.0055 | 40.8 | -15.9 | 40.8 | - | 13.5 | 40.8 | |
| 50 | DeepSeek V4 Flash 0731 (Reasoning, Max Effort) DeepSeek |
2026-07-31 | $0.140 | 5.02m | 0.0034 | 40.8 | -15.9 | 40.8 | - | 11.5 | 40.8 | |
| 51 | Gemini 3.6 Flash (high) |
2026-07-21 | $0.356 | 2.23m | 0.0088 | 40.3 | -16.5 | 40.3 | - | 15.5 | 40.3 | |
| 52 | Gemini 3.5 Flash (high) |
2026-05-19 | $1.01 | 3.23m | 0.0255 | 39.7 | -17.0 | 39.7 | - | 20.1 | 39.7 | |
| 53 | Kimi K3 (low) Kimi |
2026-07-16 | $0.252 | 1.79m | 0.0068 | 37.1 | -19.7 | 37.1 | - | 14.0 | 37.1 | |
| 54 | Gemini 3.1 Pro Preview |
2026-02-19 | $0.335 | 2.00m | 0.0091 | 36.7 | -20.1 | 36.7 | - | 15.3 | 36.7 | |
| 55 | Qwen3.7 Max Alibaba |
2026-05-19 | $0.609 | 1.96m | 0.0166 | 36.6 | -20.1 | 36.6 | - | 17.8 | 36.6 | |
| 56 | MiniMax-M3 MiniMax |
2026-06-01 | $0.229 | 5.70m | 0.0064 | 35.7 | -21.0 | 35.7 | - | 13.6 | 35.7 | |
| 57 | ✓ | DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek |
2026-04-24 | $0.0524 | 6.71m | 0.0015 | 34.5 | -22.2 | 34.5 | - | 7.2 | 34.5 |
| 58 | Quasar 438B (max, based on GLM-5.2) Multiverse Computing |
2026-08-10 | $0.902 | 5.32m | 0.0264 | 34.2 | -22.5 | 34.2 | - | 19.6 | 34.2 | |
| 59 | ✓ | MiMo-V2.5-Pro Xiaomi |
2026-04-22 | $0.0368 | 9.91m | 0.0011 | 32.6 | -24.2 | 32.6 | - | 5.7 | 32.6 |
| 60 | Hy3 Tencent |
2026-07-06 | $0.0407 | 5.09m | 0.0013 | 32.4 | -24.4 | 32.4 | - | 6.1 | 32.4 | |
| 61 | Inkling Small Thinking Machines |
2026-07-30 | $0.0719 | 3.50m | 0.0022 | 32.2 | -24.5 | 32.2 | - | 8.6 | 32.2 | |
| 62 | Inkling (xhigh) Thinking Machines |
2026-07-15 | $0.438 | 7.37m | 0.0136 | 32.2 | -24.6 | 32.2 | - | 16.4 | 32.2 | |
| 63 | Qwen3.7 Plus Alibaba |
2026-06-01 | $0.167 | 8.13m | 0.0052 | 31.9 | -24.9 | 31.9 | - | 12.2 | 31.9 | |
| 64 | DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek |
2026-04-24 | $0.0468 | - | 0.0015 | 30.7 | -26.0 | 30.7 | - | 6.7 | 30.7 | |
| 65 | Grok 4.3 (high) SpaceXAI |
2026-04-30 | $0.135 | 1.84m | 0.0045 | 29.9 | -26.8 | 29.9 | - | 11.3 | 29.9 | |
| 66 | Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA |
2026-06-04 | $0.386 | 2.58m | 0.0132 | 29.3 | -27.4 | 29.3 | - | 15.9 | 29.3 | |
| 67 | Gemini 3.5 Flash-Lite |
2026-07-21 | $0.0796 | 39s | 0.0029 | 27.6 | -29.2 | 27.6 | - | 9.0 | 27.6 | |
| 68 | ✓ | Ling 3.0 Flash InclusionAI |
2026-08-04 | $0.0306 | 1.63m | 0.0011 | 27.4 | -29.4 | 27.4 | - | 4.9 | 27.4 |
| 69 | Muse Glimmer (high) Meta |
2026-08-10 | $0.0429 | 1.95m | 0.0018 | 24.4 | -32.4 | 24.4 | - | 6.3 | 24.4 | |
| 70 | Claude 4.5 Haiku (Reasoning) Anthropic |
2025-10-15 | $0.199 | 3.07m | 0.0089 | 22.5 | -34.3 | 22.5 | - | 13.0 | 22.5 | |
| 71 | gpt-oss-120b (high) OpenAI |
2025-08-05 | $0.123 | 3.05m | 0.0079 | 15.6 | -41.2 | 15.6 | - | 10.9 | 15.6 | |
| 72 | ✓ | Granite 4.2 8B IBM |
2026-08-25 | $0.0230 | 3.59m | 0.0017 | 13.9 | -42.8 | 13.9 | - | 3.6 | 13.9 |
| 73 | Mistral Large 3 Mistral |
2025-12-02 | $0.0689 | 45s | 0.0062 | 11.1 | -45.6 | 11.1 | - | 8.4 | 11.1 | |
| 74 | ✓ | Granite 4.2 3B IBM |
2026-08-25 | $0.00552 | 58s | 0.0005 | 10.7 | -46.1 | 10.7 | - | 0.0 | 10.7 |
| 75 | Celeris-1 Celeris |
2026-07-24 | $0.181 | 6.3s | 0.0237 | 7.6 | -49.1 | 7.6 | - | 12.6 | 7.6 |