Inference runtime
vLLM
High-throughput inference and serving runtime for production model deployments.
LinuxCUDAROCm
Hardware
Supported hardware records
DGX B200 (8× B200)
NVIDIA
DGX Station (GB300)
NVIDIA
B300 SXM
NVIDIA
Instinct MI350X
AMD
Instinct MI355X
AMD
Instinct MI325X
AMD
Instinct MI300X
AMD
B200 SXM
NVIDIA
GH200 Grace Hopper (144 GB HBM3e)
NVIDIA
H200 SXM
NVIDIA · Production CUDA serving.
RTX PRO 6000 Blackwell Server Edition
NVIDIA
RTX PRO 6000 Blackwell Workstation Edition
NVIDIA
H100 NVL
NVIDIA
A100 SXM 80 GB
NVIDIA
H100 SXM
NVIDIA · Production CUDA serving.
L40S
NVIDIA
RTX 6000 Ada Generation
NVIDIA
GeForce RTX 5090
NVIDIA · CUDA serving path.
GeForce RTX 3090
NVIDIA
GeForce RTX 4090
NVIDIA · CUDA serving path.
L4 Tensor Core GPU
NVIDIA
T4 Tensor Core GPU
NVIDIA