Directory
Hardware
Compare GPU/CPU chips, prompt-processing (prefill) speed and inference generation speed. Token rates are measured for specific models and runtimes, not theoretical hardware specifications.
56 hardware models · Showing 26–50
| Select hardware to compare | Hardware | GPU / CPU chip | Prefill (tok/s) | Inference (tok/s) | RAM / VRAM |
|---|---|---|---|---|---|
| H100 NVL | — | — | 94 GB GPU | ||
| GH200 Grace Hopper (144 GB HBM3e) | — | — | 144 GB GPU | ||
| GeForce RTX 5090 | — | — | 32 GB GPU | ||
| GeForce RTX 4090 | — | — | 24 GB GPU | ||
| GeForce RTX 3090 | — | — | 24 GB GPU | ||
| Gaudi 3 AI Accelerator | — | — | 128 GB GPU | ||
| Gaudi 2 AI Accelerator | — | — | 96 GB GPU | ||
| Tesla V100 32 GB / 2× Intel Xeon Platinum 8168 | — | — | 512 GB GPU total | ||
| Tesla V100 16 GB / 2× Intel Xeon E5-2698 v4 | — | — | 128 GB GPU total | ||
| Tesla P100 16 GB / 2× Intel Xeon E5-2698 | — | — | 128 GB GPU total | ||
| NVIDIA A100 80 GB / AMD EPYC 7742 | — | — | 320 GB GPU total | ||
| NVIDIA A100 40 GB / AMD EPYC 7742 | — | — | 160 GB GPU total | ||
| Tesla V100 16 GB / Intel Xeon E5-2698 v4 | — | — | 64 GB GPU total | ||
| GB300 Blackwell Ultra / NVIDIA Grace Arm Neoverse V2 | — | — | 748 GB coherent | ||
| GB10 Blackwell / Arm Cortex-X925 + Cortex-A725 | — | — | 128 GB unified | ||
| NVIDIA H200 SXM / 2× Intel Xeon Platinum 8480C | — | — | 1,128 GB GPU total | ||
| NVIDIA H100 SXM / 2× Intel Xeon Platinum 8480C | — | — | 640 GB GPU total | ||
| GH200 Grace Hopper Superchip / 256× NVIDIA Grace | — | — | 147,456 GB coherent | ||
| NVIDIA GB300 Blackwell Ultra / 36× NVIDIA Grace | — | — | 20,000 GB GPU total | ||
| NVIDIA Blackwell (GB200) / 36 NVIDIA Grace Arm CPUs | — | — | 13,400 GB GPU total | ||
| NVIDIA B200 | — | — | 1,440 GB GPU total | ||
| NVIDIA A100 80 GB / 2× AMD EPYC 7742 | — | — | 640 GB GPU total | ||
| NVIDIA A100 40 GB / 2× AMD EPYC 7742 | — | — | 320 GB GPU total | ||
| NVIDIA GB10 Blackwell / NVIDIA Grace GB10 | — | — | 128 GB unified | ||
| B300 | — | — | 288 GB GPU |