Tulu 3 8B

About 6.1 GB at Q4, 10.3 GB at Q8 and 19 GB at FP16. These are estimates from the 8B parameter count, covering the weights plus about 15% and 0.5 GB for a short context.

Yes. A 24 GB GPU fits at FP16 (estimated 19 GB), 16 GB and 12 GB GPUs fit at Q8 (estimated 10.3 GB) and an 8 GB GPU fits at Q4 (estimated 6.1 GB).

Yes. At Q4 the estimated VRAM is 6.1 GB, and assuming macOS lets the GPU use about 75% of unified memory, the smallest common Mac memory size that fits Q4 is 16 GB.

Yes, Tulu 3 8B has open weights, so you can self-host it for full control of your data.

Run Tulu 3 8B locally

No. Tulu 3 8B accepts text only and does not read images, audio, video or PDFs.

Mistral Large 2.1 by Mistral AI, Qwen2.5-Coder-0.5B by Alibaba and Aya Expanse 32B by Cohere. They are matched on model kind and release date, with no comparable price.

Mistral Large 2.1Tulu 3 8B vs Mistral Large 2.1Qwen2.5-Coder-0.5BTulu 3 8B vs Qwen2.5-Coder-0.5BAya Expanse 32BTulu 3 8B vs Aya Expanse 32B

Open weights. Tulu 3 8B is released under the Llama 3.1 Community License.

The weights are available in the Hugging Face repository allenai/Llama-3.1-Tulu-3-8B.

Run Tulu 3 8B locally

Tulu 3 8B was made by Ai2 and released on November 21, 2024.

More from Ai2

Tulu 3 8B has open weights.