Directory
Hardware
Compare GPU/CPU chips, prompt-processing (prefill) speed and inference generation speed. Token rates are measured for specific models and runtimes, not theoretical hardware specifications.
56 hardware models · Showing 26–50
| Select hardware to compare | Hardware | GPU / CPU chip | Prefill (tok/s) | Inference (tok/s) | RAM / VRAM |
|---|---|---|---|---|---|
| Gaudi 3 AI Accelerator | — | — | 128 GB GPU | ||
| GeForce RTX 3090 | — | — | 24 GB GPU | ||
| GeForce RTX 4090 | — | — | 24 GB GPU | ||
| GeForce RTX 5090 | — | — | 32 GB GPU | ||
| GH200 Grace Hopper (144 GB HBM3e) | — | — | 144 GB GPU | ||
| H100 NVL | — | — | 94 GB GPU | ||
| H100 | — | — | 80 GB GPU | ||
| H200 | — | — | 141 GB GPU | ||
| Instinct MI300X | — | — | 192 GB GPU | ||
| Instinct MI325X | — | — | 256 GB GPU | ||
| Instinct MI350X | — | — | 288 GB GPU | ||
| MI355X | — | — | 288 GB GPU | ||
| L4 Tensor Core GPU | — | — | 24 GB GPU | ||
| L40S | — | — | 48 GB GPU | ||
| Apple M4 Pro 20-core GPU / Apple M4 Pro | — | — | 64 GB unified | ||
| Apple M4 Max 40-core GPU / Apple M4 Max | — | — | 128 GB unified | ||
| M3 Ultra integrated GPU | — | — | 512 GB unified | ||
| Apple M5 10-core GPU / Apple M5 | — | — | 32 GB unified | ||
| Apple M4 Max 40-core GPU / Apple M4 Max | — | — | 128 GB unified | ||
| Apple M5 10-core GPU / Apple M5 | — | — | 32 GB unified | ||
| Apple M5 Max 40-core GPU / Apple M5 Max | — | — | 128 GB unified | ||
| Apple M5 Pro 20-core GPU / Apple M5 Pro | — | — | 64 GB unified | ||
| Radeon AI PRO R9700 | — | — | 32 GB GPU | ||
| Radeon PRO W7900 | — | — | 48 GB GPU | ||
| RTX 6000 Ada Generation | — | — | 48 GB GPU |