Directory
Hardware
Compare GPU/CPU chips, prompt-processing (prefill) speed and inference generation speed. Token rates are measured for specific models and runtimes, not theoretical hardware specifications.
29 hardware models · Showing 26–29
| Select hardware to compare | Hardware | GPU / CPU chip | Prefill (tok/s) | Inference (tok/s) | RAM / VRAM |
|---|---|---|---|---|---|
| T4 Tensor Core GPU | — | — | 16 GB GPU | ||
| Tesla P100 SXM2 16 GB | — | — | 16 GB GPU | ||
| Tesla V100 SXM2 16 GB | — | — | 16 GB GPU | ||
| Tesla V100 SXM3 32 GB | — | — | 32 GB GPU |