SWE-bench Leaderboard
Real GitHub issues solved end-to-end. · Prices checked November 11, 2026
SWE-bench Verified gives a model a real repository and a real GitHub issue, and asks for a patch that passes the project’s existing tests. It is the hardest of the benchmarks we track and the closest to what a coding agent is actually asked to do.
Leaderboard
28 of 28 scoredLeaderboardScores as published by each benchmark. Prices checked November 11, 2026.
Claude Opus 4.8 leads on SWE-bench Verified at 88.6%. The median across the 28 models we hold a score for is 49.2%. The cheapest model on this board is DeepSeek-V3 at $0.4910/M blended, scoring 42%.
| # | Model | Provider | Score | Input /1M | Output /1M | Blended 70/30 | Context |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Anthropic | 88.6% | $5.00 | $25.00 | $11.00 | 1,000,000 |
| 2 | Claude Opus 4.5 | Anthropic | 80.9% | $5.00 | $25.00 | $11.00 | 200,000 |
| 3 | Claude Opus 4.6 | Anthropic | 80.84% | $5.00 | $25.00 | $11.00 | 1,000,000 |
| 4 | Claude Sonnet 4.6 | Anthropic | 79.6% | $3.00 | $15.00 | $6.60 | 1,000,000 |
| 5 | Claude Sonnet 4.5 | Anthropic | 77.2% | $3.00 | $15.00 | $6.60 | 200,000 |
| 6 | GPT-5 | OpenAI | 74.9% | $1.25 | $10.00 | $3.88 | 400,000 |
| 7 | Claude Haiku 4.5 | Anthropic | 73.3% | $1.00 | $5.00 | $2.20 | 200,000 |
| 8 | o3 | OpenAI | 69.1% | $2.00 | $8.00 | $3.80 | 200,000 |
| 9 | o4-mini | OpenAI | 68.1% | $1.10 | $4.40 | $2.09 | 200,000 |
| 10 | Claude Sonnet 3.7 | Anthropic | 62.3% | $3.00 | $15.00 | $6.60 | 200,000 |
| 11 | Gemini 2.5 Pro | 59.6% | $1.25 | $10.00 | $3.88 | 1,000,000 | |
| 12 | DeepSeek-R1-0528 | DeepSeek | 57.6% | $0.5500 | $2.19 | $1.04 | 64,000 |
| 13 | GPT-4.1 | OpenAI | 54.6% | $2.00 | $8.00 | $3.80 | 1,000,000 |
| 14 | DeepSeek-R1 | DeepSeek | 49.2% | $0.5500 | $2.19 | $1.04 | 64,000 |
| 15 | DeepSeek-R1 (Groq) | Groq | 49.2% | $0.7500 | $0.9900 | $0.8220 | 128,000 |
| 16 | DeepSeek-R1 (Together) | Together AI | 49.2% | $3.00 | $7.00 | $4.20 | 64,000 |
| 17 | DeepSeek-R1 (fw) | Fireworks AI | 49.2% | $3.00 | $8.00 | $4.50 | 64,000 |
| 18 | DeepSeek-R1 (DI) | DeepInfra | 49.2% | $0.5500 | $2.19 | $1.04 | 64,000 |
| 19 | DeepSeek-R1 (Cerebras) | Cerebras | 49.2% | $0.5500 | $0.9900 | $0.6820 | 64,000 |
| 20 | Claude Sonnet 3.5 | Anthropic | 49% | $3.00 | $15.00 | $6.60 | 200,000 |
| 21 | o3-mini | OpenAI | 49% | $1.10 | $4.40 | $2.09 | 200,000 |
| 22 | Gemini 2.5 Flash | 48.9% | $0.3000 | $2.50 | $0.9600 | 1,000,000 | |
| 23 | Grok 3 | xAI | 45% | $3.00 | $15.00 | $6.60 | 131,000 |
| 24 | DeepSeek-V3-0324 | DeepSeek | 44% | $0.2900 | $1.14 | $0.5450 | 128,000 |
| 25 | DeepSeek-V3 | DeepSeek | 42% | $0.3200 | $0.8900 | $0.4910 | 64,000 |
| 26 | Llama 4 Maverick | Meta | 36% | $0.3500 | $1.00 | $0.5450 | 128,000 |
| 27 | GPT-4o | OpenAI | 33% | $2.50 | $10.00 | $4.75 | 128,000 |
| 28 | GPT-4.1 mini | OpenAI | 23.6% | $0.4000 | $1.60 | $0.7600 | 1,000,000 |
23 of the 28 scores shown link to the page they were read on (underlined; hover for publisher, variant and date read). The rest have no recorded source yet and are being worked through.
Scores are much lower here than on code-generation benchmarks like HumanEval, and that gap is the point: writing a correct function is a different task from navigating an unfamiliar codebase to fix a bug.
About these scores
- Real GitHub issues solved end-to-end.
- Scores are as published by the model’s provider or a public leaderboard. Provider-reported figures are self-reported and are not independently re-run by us.
- Where a hosted variant of an open-weights model has no score of its own, it inherits the base model’s, because the weights are the same.
- Prices beside each score are the provider’s published token rates. How the blended rate is calculated.
Frequently asked questions
What is SWE-bench Verified?
A benchmark built from real GitHub issues in open-source Python projects. The model is given the repository and the issue text, and must produce a patch that makes the project’s existing tests pass. "Verified" refers to the human-validated subset, where each task has been checked to be solvable and correctly specified.
Is a higher SWE-bench score always better?
It is the best public signal for autonomous coding work, but it measures one language and one workflow. A model that scores slightly lower may still be the better choice if it is faster, cheaper, has a larger context window for your codebase, or follows your team’s conventions more reliably, none of which the benchmark captures.
Other leaderboards
Rankings that weigh price too
Leaderboards are recomputed on every deploy from the catalogue. Prices checked November 11, 2026.