- input
- $0.435
- cached
- $0.0036
- output
- $0.87
- Context
- 1.1Mtokens
- Released
- 1mo ago
- Size
- 528Bparameters
Cost/1M tokens
Benchmarks
DeepSWE is the primary coding benchmark where an exact model evaluation exists. Arena leaderboards and other sourced evaluations appear below; missing scores are not zero.
How it ranks
Against its predecessor, similarly priced models and the models ranked around it.
DeepSWE 1.1
Best pass@1 per model · Higher is better
Not evaluated in DeepSWE 1.1
Arena Text
Best Arena rating per model · Higher is better
Arena #30
- 1482
Claude Opus 4.8Nearby rank - 1481
GPT-5.5Nearby rank - 1480
MiMo-V2.6-Pro - 1478
Gemini 3.5 FlashNearby rank - 1476GPT-5.2 ChatNearby rank
- 1475
DeepSeek V4.1 FlashSimilar price - 1467MiMo-V2.5-ProPredecessor
- 1455
Qwen3.7 PlusSimilar price - 1436
MiniMax M3Similar price - 1433Gemini 3.1 Flash Lite PreviewSimilar price
+13 vs MiMo-V2.5-Pro (predecessor) · Ahead of 3 of 4 similarly priced models
Arena WebDev
Best Arena rating per model · Higher is better
Arena #27
- 1632
Hy4 previewNearby rank - 1625Claude Fable 5Nearby rank
- 1624MiMo-V2.6-Pro
- 1624
Qwen3.8 Flash NextNearby rank - 1622
GLM 5.3Nearby rank - 1621DeepSeek V4.1 FlashSimilar price
- 1590Qwen3.8 27BSimilar price
- 1484MiniMax M3Similar price
- 1479MiMo-V2.5-ProPredecessor
- 1256Gemini 3.1 Flash Lite PreviewSimilar price
+145 vs MiMo-V2.5-Pro (predecessor) · Ahead of 3 of 4 similarly priced models
Arena Vision
Best Arena rating per model · Higher is better
Arena #58
- 1263DeepSeek V4.1 FlashSimilar price
- 1262Qwen3.7 PlusSimilar price
- 1248MiMo-V2.6-FlashNearby rank
- 1247Gemini 2.5 ProNearby rank
- 1247MiMo-V2.6-Pro
- 1246Qwen3.5 397B A17BNearby rank
- 1245
Step 5 PreviewNearby rank - 1235Gemini 3.1 Flash Lite PreviewSimilar price
- 1234MiMo-V2.5Earlier version
- 1232MiniMax M3Similar price
Level with MiMo-V2.5 (earlier version, overlapping 95% CIs) · Ahead of 0 of 4 similarly priced models
Arena Search
Best Arena rating per model · Higher is better
Not ranked on Arena Search yet
Agent Arena
Best IPS score per model · Higher is better
Arena #19
- +0.053Hy4Similar price
- +0.026
Grok 4.7Nearby rank - +0.025
Inkling 2 SmallNearby rank - +0.024MiMo-V2.6-Pro
- +0.023Gemini 3.8 FlashNearby rank
- +0.022
Muse Spark 1.3Nearby rank - -0.059
Beam 1.0 501B A23BSimilar price - -0.080Qwen3.7 PlusSimilar price
- -0.105MiniMax M3Similar price
Ahead of 3 of 4 similarly priced models
Not evaluated in DeepSWE 1.1 as of 2026-09-22. Results are only shown for exact model versions in the official dataset. Official leaderboard.
All published scores
- Arena TextArena #301480
+13 vs MiMo-V2.5-Pro (predecessor)
- default148095% CI 1471 to 1490 · 4,066 votes
- Arena WebDevArena #271624
+145 vs MiMo-V2.5-Pro (predecessor)
- default162495% CI 1617 to 1632 · 8,790 votes
- Arena VisionArena #581247
Level with MiMo-V2.5 (earlier version, overlapping 95% CIs)
- default124795% CI 1233 to 1262 · 1,826 votes
- Agent ArenaArena #19+0.024
- default+0.02495% CI +0.015 to +0.033 · 5,764,669 observations
Arena ratings: Arena (lmarena-ai/leaderboard-dataset), licensed CC BY 4.0. Ratings come from pairwise human votes and are only comparable within one leaderboard; the interval is Arena's 95% confidence interval. Model names are matched to the Vertical Tabs catalog.