OLMo 2 13B

By our estimate, OLMo 2 13B needs about 10.1 GB at Q4, 17.3 GB at Q8 and 32 GB at FP16 of VRAM. That covers the weights plus headroom for a short context, and longer contexts need more.

Yes, by our estimates OLMo 2 13B fits on a 12 GB GPU at Q4 (about 10.1 GB) and at higher precision on larger cards. It does not fit fully on 8 GB GPUs.

Yes, a Mac with at least 16 GB of unified memory should run OLMo 2 13B at Q4, which needs an estimated 10.1 GB. This assumes macOS gives the GPU about 75% of unified memory.

Yes, OLMo 2 13B has open weights, so you can run it on your own hardware and keep prompts on your own machines.

Run OLMo 2 13B locally

No, OLMo 2 13B takes text input only.

Similar models from other labs include Mistral Large 2.1 (Mistral AI), Qwen2.5-Coder-0.5B (Alibaba) and DeepSeek-V3 (DeepSeek). They are the closest matches by kind and release date.

Mistral Large 2.1OLMo 2 13B vs Mistral Large 2.1Qwen2.5-Coder-0.5BOLMo 2 13B vs Qwen2.5-Coder-0.5BDeepSeek-V3OLMo 2 13B vs DeepSeek-V3

OLMo 2 13B has open weights under the Apache 2.0 license, so you can download and run it yourself.

Yes, OLMo 2 13B is released under the Apache 2.0 license, which allows commercial use. Check the license text for attribution terms.

The OLMo 2 13B weights are on Hugging Face at allenai/OLMo-2-1124-13B-Instruct.

Run OLMo 2 13B locally

OLMo 2 13B was made by Ai2. It was released on November 26, 2024.

More from Ai2

Yes, OLMo 2 13B has open weights, so you can run it on your own hardware without API fees.