- input
- $0.27
- cached
- $0.027
- output
- $1.08
- Context
- 200Ktokens
- Released
- 9mo ago
- Size
- 229Bparameters
Cost/1M tokens
Benchmarks
DeepSWE is the primary coding benchmark where an exact model evaluation exists. Arena leaderboards and other sourced evaluations appear below; missing scores are not zero.
How it ranks
Against its predecessor, similarly priced models and the models ranked around it.
DeepSWE 1.1
Best pass@1 per model · Higher is better
Not evaluated in DeepSWE 1.1
Arena Text
Best Arena rating per model · Higher is better
Arena #181
- 1435
Beam 1.0 501B A23BSimilar price - 1401
GPT-5.4 NanoSimilar price - 1393
Step 3.5 FlashNearby rank - 1392
MiMo-V2-FlashNearby rank - 1391
MiniMax M2.5 - 1390GPT-5 MiniNearby rank
- 1388
Claude Sonnet 3.7Nearby rank - 1378
GLM 4.6VSimilar price - 1358
DeepSeek V3Similar price - 1344MiniMax-M2Earlier version
+47 vs MiniMax-M2 (earlier version) · Ahead of 1 of 4 similarly priced models
Arena WebDev
Best Arena rating per model · Higher is better
Arena #111
- 1624MiMo-V2.6-ProSimilar price
- 1621DeepSeek V4.1 FlashSimilar price
- 1519GPT-5.6 LunaSimilar price
- 1437Beam 1.0 501B A23BSimilar price
- 1393Claude Sonnet 4.5Nearby rank
- 1389Claude Opus 4.1Nearby rank
- 1386MiniMax M2.5
- 1370
Gemma 4 31BNearby rank - 1369
Solar Pro 4Nearby rank - 1299MiniMax-M2Earlier version
+87 vs MiniMax-M2 (earlier version) · Ahead of 0 of 4 similarly priced models
Arena Search
Best Arena rating per model · Higher is better
Not ranked on Arena Search yet
Agent Arena
Best IPS score per model · Higher is better
Not ranked on Agent Arena yet
ARC-AGI-2
Best score per model · Higher is better
#62 of 85
- 72.9%DeepSeek V4.1 FlashSimilar price
- 42.4%
Qwen3.8 27BSimilar price - 5.7%GPT-5.4 NanoSimilar price
- 4.9%Gemini 2.5 ProNearby rank
- 4.9%MiniMax M2.5
- 4.9%o3 ProNearby rank
- 4.9%GLM 5Nearby rank
- 0.0%
Llama 4 MaverickSimilar price
Ahead of 1 of 4 similarly priced models
Terminal Bench
Best score per model · Higher is better
#26 of 49
- 61.6%GPT-5.1-Codex-MiniSimilar price
- 44.3%GPT-5-CodexNearby rank
- 43.2%
Kimi K2.5Nearby rank - 42.7%MiniMax M2.5
- 39.6%DeepSeek V3.2Similar price
- 38.0%Claude Opus 4.1Nearby rank
- 36.6%MiniMax-M2.1Predecessor
- 24.6%Qwen3.6 35B A3BSimilar price
- 17.1%Gemini 2.5 FlashSimilar price
+6.1 pts vs MiniMax-M2.1 (predecessor) · Ahead of 3 of 4 similarly priced models
Not evaluated in DeepSWE 1.1 as of 2026-09-22. Results are only shown for exact model versions in the official dataset. Official leaderboard.
All published scores
- Arena TextArena #1811391
+47 vs MiniMax-M2 (earlier version)
- default139195% CI 1387 to 1395 · 42,459 votes
- Arena WebDevArena #1111386
+87 vs MiniMax-M2 (earlier version)
- default138695% CI 1377 to 1394 · 7,568 votes
- ARC-AGI-2#62 of 854.9%
- default4.9%
- Terminal Bench#26 of 4942.7%
+6.1 pts vs MiniMax-M2.1 (predecessor)
- default42.7%cchuter
- default42.2%Terminus 2
Benchmark compilation: Epoch AI, Capabilities & benchmarking (CC BY attribution). Scores are converted to percentages and model names matched to the Vertical Tabs catalog. Individual evaluations are credited to their original sources. External results may carry separate licences and evaluation configurations; do not interpret scores from different harnesses as directly comparable.
Arena ratings: Arena (lmarena-ai/leaderboard-dataset), licensed CC BY 4.0. Ratings come from pairwise human votes and are only comparable within one leaderboard; the interval is Arena's 95% confidence interval. Model names are matched to the Vertical Tabs catalog.