Sign Up

Claude Sonnet 4

Anthropic
Announcement
Claude Sonnet 4 launch post cover
Context
200Ktokens
Released
1y ago

Benchmarks

DeepSWE is the primary coding benchmark where an exact model evaluation exists. Arena leaderboards and other sourced evaluations appear below; missing scores are not zero.

How it ranks

Against its predecessor, similarly priced models and the models ranked around it.

DeepSWE 1.1

Best pass@1 per model · Higher is better

Not evaluated in DeepSWE 1.1

Arena Text

Best Arena rating per model · Higher is better

Arena #165

  1. 1421
    R1 0528Similar release date
  2. 1405
    X-Ai/Grok-4-Fast-ReasoningNearby rank
  3. 1402
    Qwen3 235B-A22BSimilar release date
  4. 1402
    Claude Sonnet 4
  5. 1401
    GPT-5.4 NanoNearby rank
  6. 1400
    Qwen3 235B A22B Thinking 2507Nearby rank
  7. 1388
    Claude Sonnet 3.7Predecessor
  8. 1387
    Mistral Medium 3Similar release date
  9. 1347
    Qwen3 32BSimilar release date

+14 vs Claude Sonnet 3.7 (predecessor) · Ahead of 2 of 4 models released around the same time

Arena WebDev

Best Arena rating per model · Higher is better

Not ranked on Arena WebDev yet

Arena Vision

Best Arena rating per model · Higher is better

Arena #90

  1. 1247
    Gemini 2.5 ProSimilar release date
  2. 1216
    o3Similar release date
  3. 1208
    Claude Opus 4Nearby rank
  4. 1208
    Inkling SmallNearby rank
  5. 1206
    Claude Sonnet 4
  6. 1201
    GPT-4.1 MiniNearby rank
  7. 1200
    Mistral Large 3 2512Nearby rank
  8. 1193
    Claude Sonnet 3.7Predecessor
  9. 1155
    Mistral Medium 3Similar release date
  10. 1145
    Llama 4 MaverickSimilar release date

Level with Claude Sonnet 3.7 (predecessor, overlapping 95% CIs) · Ahead of 2 of 4 models released around the same time

Arena Search

Best Arena rating per model · Higher is better

Not ranked on Arena Search yet

Arena Document

Best Arena rating per model · Higher is better

Not ranked on Arena Document yet

Agent Arena

Best IPS score per model · Higher is better

Not ranked on Agent Arena yet

ARC-AGI-2

Best score per model · Higher is better

#60 of 85

  1. 8.6%
    Claude Opus 4Nearby rank
  2. 6.5%
    o3Nearby rank
  3. 5.9%
    Claude Sonnet 4
  4. 5.7%
    GPT-5.4 NanoNearby rank
  5. 4.9%
    Gemini 2.5 ProNearby rank
  6. 4.9%
    o3 ProSimilar release date
  7. 2.5%
    Gemini 2.5 Flash Preview 05-20Similar release date
  8. 1.1%
    R1 0528Similar release date
  9. 0.9%
    Claude Sonnet 3.7Predecessor
  10. 0.0%
    Magistral SmallSimilar release date

+5.0 pts vs Claude Sonnet 3.7 (predecessor) · Ahead of 4 of 4 models released around the same time

FrontierMath-2025-02-28-Private

Best score per model · Higher is better

#44 of 58

  1. 18.7%
    o3Similar release date
  2. 10.3%
    Gemini 2.5 Pro Preview 06-05Similar release date
  3. 4.5%
    GPT-4.1 MiniNearby rank
  4. 4.1%
    Claude Sonnet 3.7Predecessor
  5. 4.1%
    Claude Sonnet 4
  6. 3.8%
    GLM 4.6Nearby rank
  7. 2.4%
    GLM 4.7Nearby rank
  8. 0.3%
    Mistral Medium 3Similar release date
  9. 0.0%
    Llama 4 ScoutSimilar release date

Level with Claude Sonnet 3.7 (predecessor) · Ahead of 2 of 4 models released around the same time

FrontierMath-Tier-4-2025-07-01-Private

Best score per model · Higher is better

#39 of 46

  1. 2.1%
    GLM 4.6Similar release date
  2. 2.1%
    Gemini 2.5 Pro Preview 06-05Similar release date
  3. 2.1%
    o3Similar release date
  4. 2.1%
    GPT-5.4 MiniNearby rank
  5. 0.0%
    Claude Sonnet 3.5 v2Earlier version
  6. 0.0%
    Claude Sonnet 4
  7. 0.0%
    Kimi K2 ThinkingNearby rank
  8. 0.0%
    GPT-4.1Nearby rank
  9. 0.0%
    Qwen3 235B A22B Thinking 2507Similar release date

Level with Claude Sonnet 3.5 v2 (earlier version) · Ahead of 0 of 4 models released around the same time

GPQA diamond

Best score per model · Higher is better

#89 of 161

  1. 84.8%
    Gemini 2.5 Pro Preview 06-05Similar release date
  2. 80.1%
    Qwen3 235B A22B Thinking 2507Nearby rank
  3. 79.7%
    Claude Sonnet 3.7Predecessor
  4. 79.2%
    Claude Sonnet 4
  5. 79.0%
    Qwen3.5-9BNearby rank
  6. 78.5%
    GPT-5.4 NanoNearby rank
  7. 76.3%
    R1 0528Similar release date
  8. 63.8%
    Qwen3 14BSimilar release date
  9. 59.5%
    Mistral Medium 3Similar release date

-0.5 pts vs Claude Sonnet 3.7 (predecessor) · Ahead of 3 of 4 models released around the same time

Humanity's Last Exam

Best score per model · Higher is better

#30 of 34

  1. 20.3%
    o3Similar release date
  2. 11.0%
    Gemini 2.5 Flash Preview 05-20Similar release date
  3. 8.0%
    Claude Sonnet 3.7Predecessor
  4. 6.7%
    Claude Opus 4Nearby rank
  5. 5.7%
    Llama 4 MaverickSimilar release date
  6. 5.5%
    Claude Sonnet 4
  7. 5.4%
    GPT-4.1Nearby rank
  8. 4.5%
    Mistral Medium 3Similar release date

-2.5 pts vs Claude Sonnet 3.7 (predecessor) · Ahead of 1 of 4 models released around the same time

OSWorld

Best score per model · Higher is better

#7 of 10

  1. 63.3%
    Kimi K2.5Nearby rank
  2. 62.9%
    Claude Sonnet 4.5Similar release date
  3. 43.9%
    Claude Sonnet 4
  4. 35.8%
    Claude Sonnet 3.7Predecessor
  5. 23.0%
    o3Similar release date
  6. 5.0%
    Qwen2.5 VL 72B InstructSimilar release date

+8.1 pts vs Claude Sonnet 3.7 (predecessor) · Ahead of 2 of 3 models released around the same time

OTIS Mock AIME 2024-2025

Best score per model · Higher is better

#91 of 158

  1. 73.3%
    Qwen3 MaxNearby rank
  2. 73.3%
    Gemma 4 31BNearby rank
  3. 71.1%
    Claude Sonnet 4
  4. 71.1%
    Gemini 3.5 Flash LiteNearby rank
  5. 71.1%
    MiniMax M3Nearby rank
  6. 70.8%
    Gemini 2.5 Flash Preview 05-20Similar release date
  7. 66.4%
    R1 0528Similar release date
  8. 66.4%
    Qwen3 14BSimilar release date
  9. 57.8%
    Claude Sonnet 3.7Predecessor
  10. 32.2%
    Mistral Medium 3Similar release date

+13.3 pts vs Claude Sonnet 3.7 (predecessor) · Ahead of 4 of 4 models released around the same time

Not evaluated in DeepSWE 1.1 as of 2026-09-22. Results are only shown for exact model versions in the official dataset. Official leaderboard.

All published scores

  • Arena TextArena #1651402

    +14 vs Claude Sonnet 3.7 (predecessor)

    • default140232K, thinking95% CI 1397 to 1406 · 34,078 votes
    • default139195% CI 1387 to 1395 · 39,126 votes
    Source:Arena Text leaderboard, 2026-11-09
  • Arena VisionArena #901206

    Level with Claude Sonnet 3.7 (predecessor, overlapping 95% CIs)

    • default120632K, thinking95% CI 1189 to 1223 · 1,141 votes
    • default118995% CI 1174 to 1203 · 1,803 votes
    Source:Arena Vision leaderboard, 2026-11-09
  • ARC-AGI-2#60 of 855.9%

    +5.0 pts vs Claude Sonnet 3.7 (predecessor)

    • default5.9%16K
    • default2.1%8K
    • default1.3%
    • default0.8%1K
    Source:External evaluation via Epoch AIOriginal-source terms apply
  • FrontierMath-2025-02-28-Private#44 of 584.1%

    Level with Claude Sonnet 3.7 (predecessor)

    • default4.1%
    Source:Epoch AI evaluation
  • FrontierMath-Tier-4-2025-07-01-Private#39 of 460.0%

    Level with Claude Sonnet 3.5 v2 (earlier version)

    • default0.0%
    Source:Epoch AI evaluation
  • GPQA diamond#89 of 16179.2%

    -0.5 pts vs Claude Sonnet 3.7 (predecessor)

    • default79.2%59K
    • default78.3%32K
    • default75.8%16K
    • default66.7%
    Source:Epoch AI evaluation
  • Humanity's Last Exam#30 of 345.5%

    -2.5 pts vs Claude Sonnet 3.7 (predecessor)

    • unknown5.5%
    Source:External evaluation via Epoch AIOriginal-source terms apply
  • OSWorld#7 of 1043.9%

    +8.1 pts vs Claude Sonnet 3.7 (predecessor)

    • default43.9%claude-4-sonnet-20250514 (50 steps)
    • default41.4%claude-4-sonnet-20250514 (100 steps)
    • default31.2%claude-4-sonnet-20250514 (15 steps)
    Source:OS World WebsiteOriginal-source terms apply
  • OTIS Mock AIME 2024-2025#91 of 15871.1%

    +13.3 pts vs Claude Sonnet 3.7 (predecessor)

    • default71.1%32K
    • default68.9%59K
    • default53.3%16K
    • default28.9%
    Source:Epoch AI evaluation

Benchmark compilation: Epoch AI, Capabilities & benchmarking (CC BY attribution). Scores are converted to percentages and model names matched to the Vertical Tabs catalog. Individual evaluations are credited to their original sources. External results may carry separate licences and evaluation configurations; do not interpret scores from different harnesses as directly comparable.

Arena ratings: Arena (lmarena-ai/leaderboard-dataset), licensed CC BY 4.0. Ratings come from pairwise human votes and are only comparable within one leaderboard; the interval is Arena's 95% confidence interval. Model names are matched to the Vertical Tabs catalog.

To give you the best experience, we use cookies for personalization, insights, and performance. Learn more.