- input
- $0.50
- cached
- $0.15
- output
- $2.50
- Context
- 262Ktokens
- Released
- 9mo ago
- Size
- 1Tparameters
Cost/1M tokens
Benchmarks
DeepSWE is the primary coding benchmark where an exact model evaluation exists. Arena leaderboards and other sourced evaluations appear below; missing scores are not zero.
How it ranks
Against its predecessor, similarly priced models and the models ranked around it.
DeepSWE 1.1
Best pass@1 per model · Higher is better
Not evaluated in DeepSWE 1.1
Arena Text
Best Arena rating per model · Higher is better
Arena #81
- 1452
Gemma 4 31BNearby rank - 1450
Claude Opus 4.1Nearby rank - 1450
Kimi K2.5 - 1448
Step 5 PreviewNearby rank - 1448
MiMo-V2-ProNearby rank - 1441
GLM 4.7Similar price - 1426
Nemotron 3 UltraSimilar price - 1423
Mistral Large 4Similar price - 1421
R1 0528Similar price
Ahead of 3 of 4 similarly priced models
Arena WebDev
Best Arena rating per model · Higher is better
Arena #90
- 1632
Hy4 previewSimilar price - 1564Mistral Large 4Similar price
- 1448DeepSeek V4 Pro 0423Similar price
- 1438MiMo-V2.5Nearby rank
- 1437
Beam 1.0 501B A23BNearby rank - 1436Kimi K2.5
- 1435Gemini 3.5 Flash LiteNearby rank
- 1435GLM 5Nearby rank
- 1435GLM 4.7Similar price
Ahead of 0 of 4 similarly priced models
Arena Vision
Best Arena rating per model · Higher is better
Arena #51
- 1266Gemini 3.5 Flash LiteSimilar price
- 1256
Seed 2.0 ProNearby rank - 1255
GPT-5.6 LunaNearby rank - 1252Kimi K2.5
- 1251GPT-5.1Nearby rank
- 1250GPT-5.4 MiniNearby rank
- 1231Mistral Large 4Similar price
- 1153GLM 4.5VSimilar price
- 1121
Qwen2.5 VL 72B InstructSimilar price
Ahead of 3 of 4 similarly priced models
Arena Search
Best Arena rating per model · Higher is better
Not ranked on Arena Search yet
Arena Document
Best Arena rating per model · Higher is better
Arena #39
- 1468Gemini 3.7 FlashSimilar price
- 1448Claude Sonnet 4.5Nearby rank
- 1446Qwen3.7 PlusSimilar price
- 1438Kimi K2.5
- 1435
MiniMax M3Similar price - 1433Gemini 2.5 ProNearby rank
- 1406GLM 5V TurboSimilar price
Ahead of 1 of 4 similarly priced models
Agent Arena
Best IPS score per model · Higher is better
Not ranked on Agent Arena yet
ARC-AGI-2
Best score per model · Higher is better
#55 of 85
- 17.6%GPT-5.1Nearby rank
- 13.6%Claude Sonnet 4.5Nearby rank
- 11.8%Kimi K2.5
- 10.3%Gemini 3.5 Flash LiteNearby rank
- 9.9%GPT-5Nearby rank
- 4.9%GLM 5Similar price
- 1.3%Gemini 3 Flash PreviewSimilar price
- 1.1%R1 0528Similar price
- 0.0%GPT-4.1 MiniSimilar price
Ahead of 4 of 4 similarly priced models
Humanity's Last Exam
Best score per model · Higher is better
#15 of 34
- 44.5%Gemini 3.8 FlashSimilar price
- 27.8%GPT-5.2Nearby rank
- 25.3%GPT-5Nearby rank
- 24.4%Kimi K2.5
- 23.7%GPT-5.1Nearby rank
- 21.6%Gemini 2.5 Pro Preview 06-05Nearby rank
- 19.4%GPT-5 MiniSimilar price
- 8.3%GLM 4.5Similar price
- 4.5%Mistral Medium 3Similar price
Ahead of 3 of 4 similarly priced models
OSWorld
Best score per model · Higher is better
#4 of 10
- 72.1%Claude Sonnet 4.6Nearby rank
- 66.3%Claude Opus 4.5Nearby rank
- 63.3%Kimi K2.5
- 62.9%Claude Sonnet 4.5Nearby rank
- 43.9%Claude Sonnet 4Nearby rank
- 5.0%Qwen2.5 VL 72B InstructSimilar price
Ahead of 1 of 1 similarly priced model
SimpleQA Verified
Best score per model · Higher is better
#58 of 87
- 66.8%Gemini 3 Flash PreviewSimilar price
- 47.0%DeepSeek V4 Pro 0423Similar price
- 35.5%Claude Sonnet 4.6Nearby rank
- 34.9%Kimi K2.6Nearby rank
- 34.3%Kimi K2.5
- 34.2%GLM 5.2Nearby rank
- 34.0%GLM 5.1Nearby rank
- 32.2%GLM 4.7Similar price
- 20.0%Mistral Large 4Similar price
Ahead of 2 of 4 similarly priced models
SWE-bench Verified
Best score per model · Higher is better
#18 of 35
- 77.6%DeepSeek V4 Pro 0423Similar price
- 75.4%Gemini 3 Flash PreviewSimilar price
- 74.8%GPT-5.3-CodexNearby rank
- 74.2%GLM 5.1Nearby rank
- 73.8%Kimi K2.5
- 73.8%GPT-5.2Nearby rank
- 73.6%GPT-5Nearby rank
- 64.7%GPT-5 MiniSimilar price
- 57.9%Qwen3.6 PlusSimilar price
Ahead of 2 of 4 similarly priced models
Terminal Bench
Best score per model · Higher is better
#25 of 49
- 64.3%Gemini 3 Flash PreviewSimilar price
- 61.6%GPT-5.1-Codex-MiniSimilar price
- 57.3%
Grok 4.20Similar price - 45.1%MiniMax M2.7Nearby rank
- 44.3%GPT-5-CodexNearby rank
- 43.2%Kimi K2.5
- 42.7%MiniMax M2.5Nearby rank
- 39.6%DeepSeek V3.2Nearby rank
- 33.4%GLM 4.7Similar price
- 27.8%Kimi K2 InstructPredecessor
+15.4 pts vs Kimi K2 Instruct (predecessor) · Ahead of 1 of 4 similarly priced models
Not evaluated in DeepSWE 1.1 as of 2026-09-22. Results are only shown for exact model versions in the official dataset. Official leaderboard.
All published scores
- Arena TextArena #811450
- default1450thinking95% CI 1446 to 1453 · 73,872 votes
- default1431instant95% CI 1424 to 1437 · 8,323 votes
- Arena WebDevArena #901436
- default1436thinking95% CI 1431 to 1441 · 19,795 votes
- default1405instant95% CI 1393 to 1417 · 3,242 votes
- Arena VisionArena #511252
- default1252thinking95% CI 1246 to 1258 · 28,014 votes
- default1236instant95% CI 1225 to 1246 · 4,093 votes
- Arena DocumentArena #391438
- default1438thinking95% CI 1431 to 1444 · 22,601 votes
- ARC-AGI-2#55 of 8511.8%
- default11.8%
- Humanity's Last Exam#15 of 3424.4%
- default24.4%
- OSWorld#4 of 1063.3%
- default63.3%Kimi-K2.5
- SimpleQA Verified#58 of 8734.3%
- default34.3%
Source:Epoch AI evaluation - SWE-bench Verified#18 of 3573.8%
- default73.8%
Source:Epoch AI evaluation - Terminal Bench#25 of 4943.2%
+15.4 pts vs Kimi K2 Instruct (predecessor)
- default43.2%Terminus 2
Benchmark compilation: Epoch AI, Capabilities & benchmarking (CC BY attribution). Scores are converted to percentages and model names matched to the Vertical Tabs catalog. Individual evaluations are credited to their original sources. External results may carry separate licences and evaluation configurations; do not interpret scores from different harnesses as directly comparable.
Arena ratings: Arena (lmarena-ai/leaderboard-dataset), licensed CC BY 4.0. Ratings come from pairwise human votes and are only comparable within one leaderboard; the interval is Arena's 95% confidence interval. Model names are matched to the Vertical Tabs catalog.