CtrlAltDebrief

SWE-bench Verified benchmark

37 AI models ranked on SWE-bench Verified. Each row shows the model’s score next to its cheapest API price per 1M tokens where it has one, so you can weigh quality against cost.

Top model today

Macaron V1 Venti (Other) leads at 86%, from $1.50 per 1M input via Novita.

Data via SWE-bench →

Models ranked

37

on this benchmark

Top score

86%

current leader

Type

Single eval

benchmark

Updated

Oct 14, 2026

benchmark scores

Cost vs SWE-bench Verified

32 priced models
Most attractive quadrant (cheap + high)
= reasoning model (extended thinking)
0%20%40%60%80%$0.03$0.1$0.3$1COST / 1M INPUT TOKENS (log) →SWE-BENCH VERIFIED SCORE ↑Macaron V1 VentiDeepSeek V4 Pro 0423MiniMax M3Kimi K2.6Inkling SmallDeepSeek V4 Flash 0423MiMo-V2.5-ProHy3Qwen3.6 35B A3Bgpt-oss-120bgpt-oss-20b

Each labelled name links to that model’s page. Curated to the cost-efficiency frontier, top scorers and cheapest so labels stay readable; the full field is in the leaderboard table below (and via “select all”). Reasoning models (extended “thinking” before answering) carry a ring around the dot.

Models ranked on SWE-bench Verified

highest score first · 37 models
Maker
#
1Macaron V1 VentiOther86%
$1.50
via Novita
$4.50
via Novita
2DeepSeek V4 Pro 0423DeepSeek81%
$0.435
via Tencent
$0.87
via Tencent
3MiniMax M3MiniMax81%
$0.23
via Wandb
$0.96
via Wandb
4Kimi K2.6Moonshot80%
$0.65
via Wandb
$3.40
via Novita
5Inkling SmallThinking Machines80%
$0.45
via DeepInfra
$1.20
via Baseten
6DeepSeek V4 Flash 0423DeepSeek79%
$0.09
via DeepInfra
$0.18
via DeepInfra
7MiMo-V2.5-ProXiaomi79%
$0.435
Routed quote
$0.87
Routed quote
8Hy3Tencent78%
$0.132
Routed quote
$0.528
Routed quote
9GLM 5Zhipu78%
$0.60
via DeepInfra
$1.92
Routed quote
10InklingThinking Machines78%
$0.95
via DeepInfra
$4.05
via Baseten
11Mistral Medium 3.5 128BMistral78%
$1.50
via Scaleway
$7.50
via Scaleway
12Qwen3.6 27BAlibaba77%
$0.15
via Libertai
$0.50
via Libertai
13Qwen3.5 397B A17BAlibaba76%
$0.164
via Aihubmix
$0.986
via Aihubmix
14Muse Glimmer 30BOther76%
$0.30
via DeepInfra
$1.20
via DeepInfra
15MiniMax M2.5MiniMax76%
$0.27
Routed quote
$1.08
Routed quote
16Macaron V1 TallOther75%
$0.45
via Novita
$2.60
via Novita
17Laguna M.1Other75%
$0.20
Routed quote
$0.40
Routed quote
18Step 3.5 FlashOther74%
$0.09
Routed quote
$0.30
Routed quote
19MiniMax M2.1MiniMax74%
$0.30
via Minimax
$1.20
via Minimax
20inclusionAI: Ring-2.6-1TOther74%
n/a
n/a
21GLM 4.7Zhipu74%
$0.40
via DeepInfra
$1.75
via DeepInfra
22Qwen3.6 35B A3BAlibaba73%
$0.05
via Sail
$0.40
via Sail
23Qwen3.5-27BAlibaba72%
$0.26
via DeepInfra
$2.40
via Novita
24inclusionAI: Ling-2.6-1TOther72%
n/a
n/a
25Qwen3.5-122B-A10BAlibaba72%
$0.113
via Aihubmix
$0.901
via Aihubmix
26Kimi K2 ThinkingMoonshot71%
$0.60
via Moonshot
$1.20
via Gmi
27Laguna XS 2.1Poolside71%
$0.06
Routed quote
$0.12
Routed quote
28Kimi K2.5Moonshot71%
$0.45
via DeepInfra
$2.25
via DeepInfra
29Qwen3 Coder NextAlibaba71%
$0.12
Routed quote
$0.548
via Aihubmix
30Solar Open2 250BUpstage70%
n/a
n/a
31MiniMax M2MiniMax69%
$0.255
Routed quote
$1.02
Routed quote
32North Mini CodeCohere68%
n/a
n/a
33gpt-oss-120bOpenAI62%
$0.03
via Wandb
$0.17
via DeepInfra
34inclusionAI: Ling-2.6-flashOther61%
n/a
n/a
35gpt-oss-20bOpenAI61%
$0.0145
via Darkbloom
$0.07
via Darkbloom
36Nemotron 3 120B A12bNVIDIA60%
$0.50
via Cloudflare
$1.50
via Cloudflare
37GLM 4.7 FlashZhipu59%
$0.06
via DeepInfra
$0.40
via Amazon Bedrock
—Hy3 previewTencent74%Hugging Face did not report this score in its latest run on Oct 14, 2026
$0.18
Routed quote
$0.60
Routed quote
—Nvidia Nemotron 3 Ultra 550B A55bNVIDIA72%Hugging Face did not report this score in its latest run on Oct 14, 2026
$0.50
via DeepInfra
$2.15
via Wandb
On-demand listed rates, OpenRouter routed quotes and Vercel reseller quotes. Enterprise commitments can be materially lower. Cost per task uses best-effort token usage. An OpenRouter routed quote is the price of whichever upstream provider OpenRouter prefers today, so it moves day to day.

Frequently asked questions

What is the SWE-bench Verified benchmark?

SWE-bench Verified is a 500-task subset of SWE-bench, human-validated by OpenAI to remove unsolvable or under-specified problems. Each task is a real GitHub issue from a popular Python repository, and the model must produce a patch that makes the project's hidden test suite pass. Scored as the percentage of issues resolved.

Which AI model scores highest on SWE-bench Verified?

Macaron V1 Venti (Other) leads with 86%, from $1.50 per 1M input tokens from Novita.

How many models are ranked on SWE-bench Verified?

37 models carry a SWE-bench Verified score. Scores come from Hugging Face leaderboards; where a model has a listed API price it is drawn from LiteLLM and OpenRouter, and models with no current price are listed without one.

Other benchmarks

compare the same models on a different eval
Intelligence Indexcomposite indexCoding Indexcomposite indexAgentic Indexcomposite indexGPQA DiamondleaderboardHumanity's Last ExamleaderboardMMLU-ProleaderboardMMMU-ProleaderboardLiveCodeBenchleaderboardSciCodeleaderboardTerminalBench HardleaderboardTerminalBench v2.1leaderboardτ²-benchleaderboardτ-bench BankingleaderboardAPEX AgentsleaderboardIT-Bench SREleaderboardGDPvalleaderboardCritPtleaderboardOmniscienceleaderboardIFBenchleaderboardLong-Context Reasoningcomposite indexSWE-bench ProleaderboardAIME 2026leaderboardARC-AGI-2leaderboardAA-Briefcaseleaderboard
Benchmark scores via Hugging Face leaderboards · scores updated Oct 14, 2026 · Listed prices from LiteLLM, OpenRouter and Vercel · catalogue refreshed Oct 14, 2026 · 18 prices older, back to Jul 20, 2026

Every weekday

AI moves fast. Here's your debrief.

News, analysis, tools, and more.

For people who build with AI