Run #905

success · fetched 2026-07-13 15:00:47 · 1.28 MB raw HTML · 572 models

Open this run in comparison view · JSON results

Top quality: GPT-5.6 Sol (max) (63.4 pts)

Cost/task vs quality?

84 models · 11 on the Pareto frontier · combined quality vs Cost per Task

#?	Pareto?	Model?	Released?	Cost/task?	Time/task?	$/Q?	Qual?	ΔTop?	Intel?	Code?	Agent?	Pen?	Score?
1	✓	GPT-5.6 Sol (max) OpenAI	2026-07-09	$1.04	3.55m	0.0164	63.4	0.0	58.9	77.4	54.0	20.2	63.4
2		Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic	2026-06-09	$2.75	5.30m	0.0436	63.1	-0.4	59.9	76.5	52.8	24.4	63.1
3	✓	GPT-5.6 Terra (max) OpenAI	2026-07-09	$0.554	2.20m	0.0093	59.7	-3.8	55.0	76.7	47.4	17.4	59.7
4		Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic	2026-05-28	$1.80	6.90m	0.0304	59.0	-4.4	55.7	74.3	47.2	22.5	59.0
5		GPT-5.5 (xhigh) OpenAI	2026-04-23	$0.992	3.48m	0.0171	58.2	-5.2	54.8	74.9	44.9	20.0	58.2
6	✓	Grok 4.5 (high) SpaceXAI	2026-07-08	$0.311	1.98m	0.0054	57.3	-6.1	53.8	72.4	45.7	14.9	57.3
7		Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic	2026-06-30	$1.53	8.64m	0.0267	57.2	-6.2	53.4	71.5	46.7	21.8	57.2
8		Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic	2026-04-16	$1.97	6.13m	0.0345	57.2	-6.3	53.5	73.6	44.4	23.0	57.2
9		GPT-5.5 (high) OpenAI	2026-04-23	$0.668	2.20m	0.0119	56.1	-7.4	53.1	71.6	43.5	18.2	56.1
10		GPT-5.6 Terra (xhigh) OpenAI	2026-07-09	$0.327	1.42m	0.0059	55.6	-7.8	51.6	70.6	44.7	15.1	55.6
11		GPT-5.4 (xhigh) OpenAI	2026-03-05	$0.887	3.29m	0.0163	54.5	-8.9	51.4	71.1	41.1	19.5	54.5
12		GLM-5.2 (max) Z AI	2026-06-16	$0.376	3.08m	0.0069	54.3	-9.1	51.1	68.8	43.1	15.8	54.3
13		GPT-5.5 (medium) OpenAI	2026-04-23	$0.406	1.34m	0.0076	53.2	-10.2	50.4	71.5	37.8	16.1	53.2
14	✓	Muse Spark 1.1 (xhigh) Meta	2026-07-09	$0.261	2.73m	0.0049	53.2	-10.3	50.6	71.3	37.5	14.2	53.2
15		Gemini 3.5 Flash (high) Google	2026-05-19	$0.586	2.80m	0.0111	52.6	-10.8	50.2	70.1	37.4	17.7	52.6
16	✓	GPT-5.6 Terra (high) OpenAI	2026-07-09	$0.236	1.09m	0.0045	52.5	-11.0	49.0	67.1	41.3	13.7	52.5
17		Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic	2026-02-17	$1.14	9.64m	0.0226	50.3	-13.1	47.2	63.0	40.8	20.6	50.3
18	✓	GPT-5.6 Terra (medium) OpenAI	2026-07-09	$0.128	33s	0.0026	49.1	-14.3	45.6	64.7	37.0	11.1	49.1
19		Qwen3.7 Max Alibaba	2026-05-19	$0.985	1.60m	0.0207	47.5	-15.9	46.0	66.0	30.6	19.9	47.5
20		Claude Sonnet 5 (Non-reasoning, High Effort) Anthropic	2026-06-30	$0.375	1.46m	0.0079	47.3	-16.2	41.7	66.4	33.7	15.7	47.3
21	✓	DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek	2026-04-24	$0.0448	8.04m	0.0010	46.7	-16.8	44.3	59.4	36.4	6.5	46.7
22		MiniMax-M3 MiniMax	2026-06-01	$0.125	3.55m	0.0027	46.1	-17.3	44.4	58.6	35.4	11.0	46.1
23		Gemini 3.1 Pro Preview Google	2026-02-19	$0.291	1.52m	0.0064	45.6	-17.9	46.5	68.8	21.4	14.6	45.6
24		GPT-5.5 (low) OpenAI	2026-04-23	$0.211	36s	0.0047	44.9	-18.5	43.5	60.9	30.4	13.2	44.9
25	✓	MiMo-V2.5-Pro Xiaomi	2026-04-22	$0.0314	5.50m	0.0007	43.8	-19.6	42.2	60.2	29.1	5.0	43.8
26		GPT-5.6 Terra (low) OpenAI	2026-07-09	$0.101	20s	0.0023	43.1	-20.4	40.5	58.1	30.6	10.0	43.1
27	✓	DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek	2026-04-24	$0.0223	6.55m	0.0005	42.5	-20.9	40.3	56.2	31.1	3.5	42.5
28		GPT-5.4 mini (xhigh) OpenAI	2026-03-17	$0.452	7.70m	0.0107	42.1	-21.4	40.0	56.1	30.2	16.6	42.1
29		GLM-5.1 (Reasoning) Z AI	2026-04-07	$0.247	4.80m	0.0059	41.9	-21.5	40.2	55.8	29.9	13.9	41.9
30		Qwen3.6 Plus Alibaba	2026-04-02	$0.314	6.40m	0.0077	40.5	-22.9	39.6	54.5	27.6	15.0	40.5
31		Grok Build 0.1 0616 SpaceXAI	-	$0.212	3.44m	0.0053	39.8	-23.7	39.8	51.5	28.0	13.3	39.8
32		Qwen3.6 27B (Reasoning) Alibaba	2026-04-22	$0.265	6.74m	0.0068	39.3	-24.2	37.1	53.7	27.0	14.2	39.3
33	✓	MiMo-V2.5 Xiaomi	2026-04-22	$0.00975	3.09m	0.0002	39.2	-24.2	37.2	56.8	23.7	0.0	39.2
34		GPT-5.5 (Non-reasoning) OpenAI	2026-04-23	$0.171	26s	0.0044	39.2	-24.2	35.4	56.5	25.8	12.3	39.2
35		Qwen3.7 Plus Alibaba	2026-06-01	$0.195	6.64m	0.0051	38.5	-24.9	39.0	55.9	20.8	12.9	38.5
36		Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA	2026-06-04	$0.234	1.79m	0.0061	38.1	-25.3	37.8	49.3	27.4	13.7	38.1
37		Claude 4.5 Sonnet (Reasoning) Anthropic	2025-09-29	$0.413	4.97m	0.0110	37.7	-25.7	36.4	52.1	24.6	16.2	37.7
38		GLM-4.7 (Reasoning) Z AI	2025-12-22	$0.297	2.48m	0.0086	34.8	-28.6	33.7	45.3	25.4	14.7	34.8
39		Grok 4.3 (high) SpaceXAI	2026-04-30	$0.139	2.14m	0.0040	34.6	-28.8	37.6	42.2	24.1	11.4	34.6
40		Qwen3.5 397B A17B (Reasoning) Alibaba	2026-02-16	$0.333	3.87m	0.0098	33.9	-29.5	33.7	48.2	19.8	15.2	33.9
41		Qwen3.6 27B (Non-reasoning) Alibaba	2026-04-22	$0.359	3.26m	0.0107	33.5	-30.0	30.5	46.6	23.3	15.6	33.5
42		Qwen3.5 122B A10B (Reasoning) Alibaba	2026-02-24	$0.241	1.78m	0.0073	32.9	-30.5	32.3	45.7	20.7	13.8	32.9
43		GPT-5 (high) OpenAI	2025-08-07	$0.238	2.82m	0.0073	32.7	-30.7	34.7	37.8	25.7	13.8	32.7
44		Mistral Medium 3.5 Mistral	2026-04-29	$1.20	2.69m	0.0374	31.9	-31.5	29.9	46.9	19.0	20.8	31.9
45		Qwen3.6 35B A3B (Reasoning) Alibaba	2026-04-16	$0.179	2.53m	0.0056	31.6	-31.8	31.6	41.9	21.4	12.5	31.6
46		Ring-2.6-1T InclusionAI	2026-05-08	$0.345	3.21m	0.0112	30.8	-32.7	30.6	42.8	18.9	15.4	30.8
47		GLM-4.6 (Reasoning) Z AI	2025-09-30	$0.283	6.27m	0.0092	30.7	-32.7	28.7	45.8	17.7	14.5	30.7
48		Claude 4.5 Haiku (Reasoning) Anthropic	2025-10-15	$0.272	2.81m	0.0091	30.0	-33.5	29.6	43.9	16.4	14.3	30.0
49		Qwen3.5 122B A10B (Non-reasoning) Alibaba	2026-02-24	$0.177	52s	0.0061	28.9	-34.5	27.6	43.3	15.8	12.5	28.9
50		Grok 4.3 (Non-reasoning) SpaceXAI	2026-04-30	$0.293	1.31m	0.0106	27.6	-35.8	24.8	35.2	22.8	14.7	27.6
51		GPT-5.5 Instant (June 2026) OpenAI	2026-06-25	$0.711	-	0.0269	26.4	-37.0	28.9	39.4	11.0	18.5	26.4
52		Gemma 4 26B A4B (Reasoning) Google	2026-04-02	$0.0294	-	0.0012	25.3	-38.1	25.7	39.3	11.0	4.7	25.3
53		NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA	2026-03-11	$0.206	1.68m	0.0086	23.9	-39.5	25.4	37.7	8.7	13.1	23.9
54		gpt-oss-120b (high) OpenAI	2025-08-05	$0.0607	1.89m	0.0027	22.5	-40.9	23.8	30.4	13.2	7.8	22.5
55		Gemini 2.5 Pro Google	2025-06-05	$0.198	1.36m	0.0090	22.1	-41.4	25.8	33.3	7.1	13.0	22.1
56		Qwen3 Coder Next Alibaba	2026-02-03	$0.329	2.35m	0.0149	22.0	-41.4	21.1	36.2	8.8	15.2	22.0
57		Gemini 3.1 Flash-Lite Google	2026-03-03	$0.0416	1.08m	0.0019	22.0	-41.5	25.0	34.7	6.2	6.2	22.0
58		Gemma 4 31B (Non-reasoning) Google	2026-04-02	$0.0343	1.37m	0.0016	22.0	-41.5	21.8	33.2	11.0	5.4	22.0
59		Nova 2.0 Pro Preview (medium) Amazon	2025-11-27	$0.173	1.51m	0.0083	20.9	-42.5	21.8	34.0	7.0	12.4	20.9
60		Qwen3.5 9B (Reasoning) Alibaba	2026-03-02	$0.164	7.63m	0.0085	19.2	-44.3	21.4	28.7	7.4	12.1	19.2
61		Nova 2.0 Pro Preview (low) Amazon	2025-11-27	$0.213	39s	0.0123	17.3	-46.1	19.6	25.9	6.4	13.3	17.3
62		HyperNova 60B 2605 Multiverse Computing	2026-05-26	$0.0180	1.33m	0.0011	15.9	-47.5	17.8	23.2	6.7	2.5	15.9
63		DeepSeek R1 (Jan '25) DeepSeek	2025-01-20	$0.247	-	0.0160	15.4	-48.0	18.5	24.6	3.1	13.9	15.4
64		Mistral Small 3.1 Mistral	2025-03-17	$0.0410	29s	0.0027	15.4	-48.0	14.7	26.3	5.2	6.1	15.4
65		Nova 2.0 Lite (high) Amazon	2025-10-29	$0.245	4.69m	0.0166	14.8	-48.6	18.2	23.0	3.1	13.9	14.8
66		Mistral Large 3 Mistral	2025-12-02	$0.0599	1.01m	0.0043	13.8	-49.6	15.9	20.1	5.5	7.8	13.8
67		Mistral Medium 3.1 Mistral	2025-08-12	$0.145	59s	0.0105	13.8	-49.6	14.7	20.5	6.2	11.6	13.8
68		Magistral Medium 1.2 Mistral	2025-09-18	$0.747	3.37m	0.0546	13.7	-49.7	17.9	21.3	1.9	18.7	13.7
69		DeepSeek V3 (Dec '24) DeepSeek	2024-12-26	$0.0254	-	0.0020	12.9	-50.5	14.2	23.0	1.6	4.0	12.9
70		gpt-oss-20b (high) OpenAI	2025-08-05	$0.0178	1.38m	0.0014	12.9	-50.5	14.9	20.7	3.1	2.5	12.9
71		DeepSeek V3 0324 DeepSeek	2025-03-25	$0.190	-	0.0149	12.7	-50.7	15.4	21.2	1.5	12.8	12.7
72		Nova 2.0 Pro Preview (Non-reasoning) Amazon	2025-11-27	$0.246	30s	0.0194	12.7	-50.7	14.4	20.9	2.9	13.9	12.7
73		GPT-4.1 mini OpenAI	2025-04-14	$0.0637	39s	0.0052	12.2	-51.2	14.8	20.2	1.7	8.0	12.2
74		Qwen3 Next 80B A3B (Reasoning) Alibaba	2025-09-11	$0.169	1.82m	0.0140	12.1	-51.4	16.7	17.4	2.1	12.3	12.1
75		Llama 4 Maverick Meta	2025-04-05	$0.0316	18s	0.0030	10.6	-52.8	14.3	16.3	1.3	5.0	10.6
76		NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA	2025-12-15	$0.0183	4.14m	0.0018	10.2	-53.2	14.2	14.4	2.0	2.6	10.2
77		Qwen3 30B A3B 2507 (Reasoning) Alibaba	2025-07-30	$0.0716	2.63m	0.0076	9.4	-54.0	14.4	12.1	1.8	8.6	9.4
78		Ministral 3 14B Mistral	2025-12-02	$0.149	2.22m	0.0162	9.2	-54.2	11.1	14.4	2.2	11.7	9.2
79		Mistral Small 3.2 Mistral	2025-06-20	$0.124	52s	0.0148	8.4	-55.1	10.6	12.5	2.0	10.9	8.4
80		GPT-4.1 nano OpenAI	2025-04-14	$0.0399	40s	0.0055	7.3	-56.1	9.6	11.1	1.2	6.0	7.3
81		Llama 3.3 Instruct 70B Meta	2024-12-06	$0.0798	19s	0.0110	7.2	-56.2	9.4	11.9	0.3	9.0	7.2
82		Ministral 3 8B Mistral	2025-12-02	$0.185	2.33m	0.0279	6.6	-56.8	9.0	9.7	1.2	12.7	6.6
83	✓	Llama 4 Scout Meta	2025-04-05	$0.00851	11s	0.0013	6.4	-57.0	10.0	8.2	1.1	0.0	6.4
84		Ministral 3 3B Mistral	2025-12-02	$0.126	1.87m	0.0288	4.4	-59.0	6.8	4.8	1.6	11.0	4.4