OLMo 2 7B

OLMo 2 7B needs an estimated 5.6 GB of VRAM at Q4, 9.4 GB at Q8 and 17.3 GB at FP16, all derived from the 7.3B parameter count. Figures cover weights plus about 15% and 0.5 GB for a short context.

Yes. OLMo 2 7B fits on a 24 GB GPU at FP16 (estimated 17.3 GB), on 16 GB and 12 GB GPUs at Q8 (estimated 9.4 GB) and on an 8 GB GPU at Q4 (estimated 5.6 GB).

Yes. With Q4 needing an estimated 5.6 GB and the assumption that macOS lets the GPU use about 75% of unified memory, the smallest common Mac memory size that fits Q4 is 8 GB (estimate).

Yes, OLMo 2 7B has open weights.

Run OLMo 2 7B locally

No, OLMo 2 7B only supports text as an input modality.

The closest alternatives are Mistral Large 2.1 by Mistral AI, Qwen2.5-Coder-0.5B by Alibaba and DeepSeek-V3 by DeepSeek. They are matched on model kind and release date, with no comparable price.

Mistral Large 2.1OLMo 2 7B vs Mistral Large 2.1Qwen2.5-Coder-0.5BOLMo 2 7B vs Qwen2.5-Coder-0.5BDeepSeek-V3OLMo 2 7B vs DeepSeek-V3

OLMo 2 7B is open weights, licensed under Apache 2.0.

Yes, OLMo 2 7B is licensed under Apache 2.0, a permissive license that allows commercial use.

The OLMo 2 7B weights are available in the Hugging Face repository allenai/OLMo-2-1124-7B-Instruct.

Run OLMo 2 7B locally

OLMo 2 7B was made by Ai2 and released on November 26, 2024.

More from Ai2

Yes, OLMo 2 7B is open weights.