Sign Up

Decision support

Claude Sonnet 4 vs Apple On-Device Model (2025)

Compare canonical objects directly. The useful dimensions change with the object type, so model and hardware comparisons stay separate.

ModelsHardware
Claude Sonnet 4

Anthropic

Sources:models.dev

Apple On-Device Model (2025)

Apple

Benchmark results

  • Arena Text

    Arena rating · Higher is better

    • Claude Sonnet 4
      1402
      Arena #165CI 1397 to 1406
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default140232K, thinking · 95% CI 1397 to 1406 · 34,078 votes
    • default139195% CI 1387 to 1395 · 39,126 votes
    Apple On-Device Model (2025)

    Not reported

    Source:Arena Text leaderboard, 2026-11-09

  • Arena Vision

    Arena rating · Higher is better

    • Claude Sonnet 4
      1206
      Arena #90CI 1189 to 1223
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default120632K, thinking · 95% CI 1189 to 1223 · 1,141 votes
    • default118995% CI 1174 to 1203 · 1,803 votes
    Apple On-Device Model (2025)

    Not reported

    Source:Arena Vision leaderboard, 2026-11-09

  • ARC-AGI-2

    Best published score · Higher is better

    • Claude Sonnet 4
      5.9%
      #60 of 85
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default5.9%16K
    • default2.1%8K
    • default1.3%
    • default0.8%1K
    Apple On-Device Model (2025)

    Not reported

    Source:External evaluation via Epoch AIOriginal-source terms apply

  • FrontierMath-2025-02-28-Private

    Best published score · Higher is better

    • Claude Sonnet 4
      4.1%
      #44 of 58
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default4.1%
    Apple On-Device Model (2025)

    Not reported

    Source:Epoch AI evaluation

  • FrontierMath-Tier-4-2025-07-01-Private

    Best published score · Higher is better

    • Claude Sonnet 4
      0.0%
      #39 of 46
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default0.0%
    Apple On-Device Model (2025)

    Not reported

    Source:Epoch AI evaluation

  • GPQA diamond

    Best published score · Higher is better

    • Claude Sonnet 4
      79.2%
      #89 of 161
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default79.2%59K
    • default78.3%32K
    • default75.8%16K
    • default66.7%
    Apple On-Device Model (2025)

    Not reported

    Source:Epoch AI evaluation

  • Humanity's Last Exam

    Best published score · Higher is better

    • Claude Sonnet 4
      5.5%
      #30 of 34
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • unknown5.5%
    Apple On-Device Model (2025)

    Not reported

    Source:External evaluation via Epoch AIOriginal-source terms apply

  • OSWorld

    Best published score · Higher is better

    • Claude Sonnet 4
      43.9%
      #7 of 10
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default43.9%claude-4-sonnet-20250514 (50 steps)
    • default41.4%claude-4-sonnet-20250514 (100 steps)
    • default31.2%claude-4-sonnet-20250514 (15 steps)
    Apple On-Device Model (2025)

    Not reported

    Source:OS World WebsiteOriginal-source terms apply

  • OTIS Mock AIME 2024-2025

    Best published score · Higher is better

    • Claude Sonnet 4
      71.1%
      #91 of 158
    • Apple On-Device Model (2025)
      Not reported
    Runs and sources
    Claude Sonnet 4
    • default71.1%32K
    • default68.9%59K
    • default53.3%16K
    • default28.9%
    Apple On-Device Model (2025)

    Not reported

    Source:Epoch AI evaluation

Each model's best published result is shown; "Not reported" means no matched evaluation, not a score of zero. Arena and DeepSWE ranks are the leaderboards' own; other ranks are among catalogue models with a score. Results from different harnesses or settings are not directly comparable.

Original-source terms apply to evaluations credited to their original publishers.

Benchmark compilation: Epoch AI, Capabilities & benchmarking (CC BY attribution). Scores are converted to percentages and model names matched to the Vertical Tabs catalog. Individual evaluations are credited to their original sources. External results may carry separate licences and evaluation configurations; do not interpret scores from different harnesses as directly comparable.

Arena ratings: Arena (lmarena-ai/leaderboard-dataset), licensed CC BY 4.0. Ratings come from pairwise human votes and are only comparable within one leaderboard; the interval is Arena's 95% confidence interval. Model names are matched to the Vertical Tabs catalog.

Model specifications

Release date
May 22, 2025
June 9, 2025
Context window
200K
—
Native context
—
—
Max output tokens
64,000
—
Knowledge cutoff
2025-03-31
—
Open weights
Closed
Closed
Parameter count
—
3B

API pricing · USD per 1M tokens

Input
—
—
Output
—
—
Cached input
—
—
Cache write
—
—

Capabilities and modalities

Input modalities
text, image, file
text, image
Output modalities
text
text
Tool calling
Supported
—
Reasoning option
Supported
—
Supported options
tools, reasoning, temperature
—
Capabilities
—
—

Architecture and availability

Architecture
—
—
Model families
—
—
Tokenizer
—
—
Instruction format
—
—
License
—
proprietary
Providers
0
0
Provider endpoints
0
0
Ollama variants
0
0
Metadata source
ai-research
curated

A dash means the field is not reported. Provider-specific performance and pricing may differ.

To give you the best experience, we use cookies for personalization, insights, and performance. Learn more.